[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:08 dist-test-slave-5bn2 krb5kdc[27138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:08 dist-test-slave-5bn2 krb5kdc[27138](info): set up 2 sockets
Aug 11 22:21:08 dist-test-slave-5bn2 krb5kdc[27138](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 22:21:09.507709 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:09 dist-test-slave-5bn2 krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:09.646205 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:46639
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:43577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:46639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:09.762771 27162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:09.763157 27162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:09.763211 27162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:09.767225 27162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:09.767288 27162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:09.767306 27162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:09.767323 27162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:09.767338 27162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:09.772809 27162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:43577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:46639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:46639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:09.773980 27162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:09.775032 27162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:09.783934 27173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:09.784022 27175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:09.783952 27180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:09.784159 27162 server_base.cc:1034] running on GCE node
I20260811 22:21:09.784914 27162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:09.786145 27162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:09.787312 27162 hybrid_clock.cc:648] HybridClock initialized: now 1786486869787299 us; error 112 us; skew 500 ppm
Aug 11 22:21:09 dist-test-slave-5bn2 krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486869, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:09.790503 27162 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:09.791594 27162 webserver.cc:460] Webserver started at http://127.26.125.62:38709/ using document root <none> and password file <none>
I20260811 22:21:09.792174 27162 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:09.792243 27162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:09.792477 27162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:09.794828 27162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "dccf1ba401c748908a1615175d78c3d3"
format_stamp: "Formatted at 2026-08-11 22:21:09 on dist-test-slave-5bn2"
server_key: "620935aa9050ea44e34cdc713b115148"
server_key_iv: "869a941adc7f8d8601f785fb8ecc409a"
server_key_version: "encryptionkey@0"
I20260811 22:21:09.795511 27162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "dccf1ba401c748908a1615175d78c3d3"
format_stamp: "Formatted at 2026-08-11 22:21:09 on dist-test-slave-5bn2"
server_key: "620935aa9050ea44e34cdc713b115148"
server_key_iv: "869a941adc7f8d8601f785fb8ecc409a"
server_key_version: "encryptionkey@0"
I20260811 22:21:09.799623 27162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:09.802409 27190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:09.804085 27162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:21:09.804227 27162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "dccf1ba401c748908a1615175d78c3d3"
format_stamp: "Formatted at 2026-08-11 22:21:09 on dist-test-slave-5bn2"
server_key: "620935aa9050ea44e34cdc713b115148"
server_key_iv: "869a941adc7f8d8601f785fb8ecc409a"
server_key_version: "encryptionkey@0"
I20260811 22:21:09.804354 27162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:09.823695 27162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:09.827577 27162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:09.827781 27162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:09.835280 27162 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:46639
I20260811 22:21:09.835302 27274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:46639 every 8 connection(s)
I20260811 22:21:09.836336 27162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:09.839278 27275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:09.845391 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27162
I20260811 22:21:09.845644 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:09.845865 27275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3: Bootstrap starting.
I20260811 22:21:09.847536 27124 external_mini_cluster.cc:1428] Setting key 48231f80ba7ac06ec966f65b113b7b62
I20260811 22:21:09.849349 27275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:09.850359 27275 log.cc:824] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:09.852927 27275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3: No bootstrap required, opened a new log
I20260811 22:21:09.856917 27275 raft_consensus.cc:348] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf1ba401c748908a1615175d78c3d3" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46639 } }
I20260811 22:21:09.857218 27275 raft_consensus.cc:374] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:09.857317 27275 raft_consensus.cc:729] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dccf1ba401c748908a1615175d78c3d3, State: Initialized, Role: FOLLOWER
I20260811 22:21:09.857801 27275 consensus_queue.cc:260] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf1ba401c748908a1615175d78c3d3" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46639 } }
I20260811 22:21:09.857942 27275 raft_consensus.cc:388] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:09.858022 27275 raft_consensus.cc:482] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:09.858124 27275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:09.859097 27275 raft_consensus.cc:504] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf1ba401c748908a1615175d78c3d3" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46639 } }
I20260811 22:21:09.859431 27275 leader_election.cc:302] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dccf1ba401c748908a1615175d78c3d3; no voters:
I20260811 22:21:09.859736 27275 leader_election.cc:288] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:09.859823 27281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:09.860018 27281 raft_consensus.cc:686] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [term 1 LEADER]: Becoming Leader. State: Replica: dccf1ba401c748908a1615175d78c3d3, State: Running, Role: LEADER
I20260811 22:21:09.860387 27281 consensus_queue.cc:237] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf1ba401c748908a1615175d78c3d3" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46639 } }
I20260811 22:21:09.861135 27275 sys_catalog.cc:564] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:09.862165 27282 sys_catalog.cc:455] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dccf1ba401c748908a1615175d78c3d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf1ba401c748908a1615175d78c3d3" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46639 } } }
I20260811 22:21:09.862306 27282 sys_catalog.cc:458] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:09.862620 27283 sys_catalog.cc:455] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dccf1ba401c748908a1615175d78c3d3. Latest consensus state: current_term: 1 leader_uuid: "dccf1ba401c748908a1615175d78c3d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf1ba401c748908a1615175d78c3d3" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46639 } } }
I20260811 22:21:09.862785 27283 sys_catalog.cc:458] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3 [sys.catalog]: This master's current role is: LEADER
W20260811 22:21:09.866583 27305 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:09.866655 27305 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:09.866730 27295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:09.867789 27295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 22:21:09 dist-test-slave-5bn2 krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:09.874135 27295 catalog_manager.cc:1347] Generated new cluster ID: 0c3234a6c9ec4dacb36dbd60fc7aa09a
I20260811 22:21:09.874280 27295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:09.876309 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:09.851669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.26.125.62:46639)
0811 22:21:09.852180 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:09.852202 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:09.853550 (+ 1348us) server_negotiation.cc:407] Connection header received
0811 22:21:09.856494 (+ 2944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:09.856512 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:09.856832 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:09.857194 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:09.860014 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:09.861278 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:09.863328 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:09.864063 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:09.864233 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:09.869680 (+ 5447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:09.869688 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:09.869705 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:09.869743 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:09.872010 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:09.872709 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:09.872715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:09.872720 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:09.872863 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:09.873264 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:09.873267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:09.873268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:09.873833 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:09.874078 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:09.875039 (+ 961us) server_negotiation.cc:299] Negotiation successful
0811 22:21:09.875368 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":127,"threads_started":1}
I20260811 22:21:09.889439 27295 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:09.890542 27295 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:09.896827 27295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dccf1ba401c748908a1615175d78c3d3: Generated new TSK 0
I20260811 22:21:09.897563 27295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:09.929486 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46639
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:43577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:10.048141 27326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:10.048481 27326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:10.048530 27326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:10.052791 27326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:10.052870 27326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:10.052945 27326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:10.058643 27326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:43577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:10.059870 27326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:10.060946 27326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:10.070951 27340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:10.071131 27326 server_base.cc:1034] running on GCE node
W20260811 22:21:10.070956 27337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:10.070979 27338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:10.071794 27326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:10.072495 27326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:10.073683 27326 hybrid_clock.cc:648] HybridClock initialized: now 1786486870073655 us; error 37 us; skew 500 ppm
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:10.076807 27326 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:10.077832 27326 webserver.cc:460] Webserver started at http://127.26.125.1:40689/ using document root <none> and password file <none>
I20260811 22:21:10.078430 27326 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:10.078498 27326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:10.078675 27326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:10.080928 27326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "b2d4c9d9c4f747e390425c8c52e74a43"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "79bb64f42273d1520200a34d4400c45d"
server_key_iv: "7a14e4fd2b2dec6eec7b96fa8f8e0db9"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.081512 27326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "b2d4c9d9c4f747e390425c8c52e74a43"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "79bb64f42273d1520200a34d4400c45d"
server_key_iv: "7a14e4fd2b2dec6eec7b96fa8f8e0db9"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.085858 27326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:10.088614 27347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.089970 27326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:10.090116 27326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "b2d4c9d9c4f747e390425c8c52e74a43"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "79bb64f42273d1520200a34d4400c45d"
server_key_iv: "7a14e4fd2b2dec6eec7b96fa8f8e0db9"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.090225 27326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:10.107725 27326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:10.111689 27326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:10.111873 27326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:10.112521 27326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:10.113420 27326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:10.113469 27326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.113512 27326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:10.113533 27326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.122982 27326 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:42261
I20260811 22:21:10.123030 27540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:42261 every 8 connection(s)
I20260811 22:21:10.124043 27326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:10.127595 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27326
I20260811 22:21:10.127712 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:10.127969 27124 external_mini_cluster.cc:1428] Setting key 53914ede0859fb78282a89676e2aee77
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:21:10.139743 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:10.126175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:37549 (local address 127.26.125.62:46639)
0811 22:21:10.126317 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:10.126321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:10.127578 (+ 1257us) server_negotiation.cc:407] Connection header received
0811 22:21:10.128661 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:10.128665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:10.128726 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:10.128826 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:10.130413 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.131007 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.132205 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.132461 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.132554 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:10.135595 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:10.135599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:10.135601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:10.135633 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:10.137165 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.137867 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.137870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.137872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.137964 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.138387 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.138389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.138390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.138587 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:10.138659 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:10.139475 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 22:21:10.139609 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:21:10.140645 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:10.126580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:46639 (local address 127.26.125.1:37549)
0811 22:21:10.127375 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:10.127426 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:10.128381 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:10.129072 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:10.129082 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:10.129642 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:10.130241 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:10.130257 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.131135 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.131138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:10.132075 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:10.132084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.132622 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.132626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:10.132801 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:10.133468 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:10.133488 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:10.135334 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:10.137311 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:10.137320 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:10.137342 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:10.137742 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:10.138066 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:10.138070 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:10.138071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:10.138291 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:10.138690 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:10.138698 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:10.139159 (+ 461us) client_negotiation.cc:769] Sending connection context
0811 22:21:10.139366 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:21:10.139680 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:21:10.142031 27541 heartbeater.cc:347] Connected to a master server at 127.26.125.62:46639
I20260811 22:21:10.142310 27541 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:10.142886 27541 heartbeater.cc:511] Master 127.26.125.62:46639 requested a full tablet report, sending...
I20260811 22:21:10.144515 27214 ts_manager.cc:194] Registered new tserver with Master: b2d4c9d9c4f747e390425c8c52e74a43 (127.26.125.1:42261)
I20260811 22:21:10.145689 27214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:37549
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:10.158457 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46639
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:43577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:10.279668 27549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:10.279981 27549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:10.280030 27549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:10.284143 27549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:10.284230 27549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:10.284300 27549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:10.289920 27549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:43577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:10.291157 27549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:10.292220 27549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:10.302623 27556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:10.302613 27555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:10.302608 27559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:10.302778 27549 server_base.cc:1034] running on GCE node
I20260811 22:21:10.303467 27549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:10.304193 27549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:10.305392 27549 hybrid_clock.cc:648] HybridClock initialized: now 1786486870305364 us; error 42 us; skew 500 ppm
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:10.308977 27549 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:10.310043 27549 webserver.cc:460] Webserver started at http://127.26.125.2:45971/ using document root <none> and password file <none>
I20260811 22:21:10.310706 27549 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:10.310774 27549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:10.311007 27549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:10.313195 27549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "83fb21ee273c43a2b84afb00f770bf69"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "6044bac159df59584fe88f192114cfa2"
server_key_iv: "cff4f7f1ae0508069ee3df728e80f537"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.313752 27549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "83fb21ee273c43a2b84afb00f770bf69"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "6044bac159df59584fe88f192114cfa2"
server_key_iv: "cff4f7f1ae0508069ee3df728e80f537"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.317749 27549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:21:10.320499 27571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.321822 27549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:10.321946 27549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "83fb21ee273c43a2b84afb00f770bf69"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "6044bac159df59584fe88f192114cfa2"
server_key_iv: "cff4f7f1ae0508069ee3df728e80f537"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.322055 27549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:10.337703 27549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:10.341241 27549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:10.341454 27549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:10.342123 27549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:10.343071 27549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:10.343124 27549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.343182 27549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:10.343206 27549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.352404 27549 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:39679
I20260811 22:21:10.352545 27757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:39679 every 8 connection(s)
I20260811 22:21:10.353466 27549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:10.357034 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27549
I20260811 22:21:10.357168 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:10.357496 27124 external_mini_cluster.cc:1428] Setting key 4a6e90eb73f5737265c2a5330b3ee588
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:10.369254 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:10.355675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:36317 (local address 127.26.125.62:46639)
0811 22:21:10.355840 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:10.355845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:10.356883 (+ 1038us) server_negotiation.cc:407] Connection header received
0811 22:21:10.358422 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:10.358426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:10.358475 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:10.358561 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:10.360094 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.360612 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.361999 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.362260 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.362386 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:10.365282 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:10.365286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:10.365288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:10.365318 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:10.366874 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.367500 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.367502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.367504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.367589 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.368015 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.368019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.368020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.368246 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:10.368359 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:10.369013 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 22:21:10.369128 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:21:10.370217 27767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:10.356139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:46639 (local address 127.26.125.2:36317)
0811 22:21:10.356668 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:10.356741 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:10.358221 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:10.358706 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:10.358714 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:10.359223 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:10.359937 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:10.359958 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.360736 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.360740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:10.361861 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:10.361880 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.362421 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.362426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:10.362604 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:10.363225 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:10.363244 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:10.365068 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:10.367000 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:10.367007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:10.367026 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:10.367391 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:10.367683 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:10.367686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:10.367689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:10.367889 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:10.368363 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:10.368371 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:10.368747 (+ 376us) client_negotiation.cc:769] Sending connection context
0811 22:21:10.368927 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 22:21:10.369212 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:21:10.371563 27759 heartbeater.cc:347] Connected to a master server at 127.26.125.62:46639
I20260811 22:21:10.371826 27759 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:10.372351 27759 heartbeater.cc:511] Master 127.26.125.62:46639 requested a full tablet report, sending...
I20260811 22:21:10.373564 27214 ts_manager.cc:194] Registered new tserver with Master: 83fb21ee273c43a2b84afb00f770bf69 (127.26.125.2:39679)
I20260811 22:21:10.374141 27214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:36317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:10.389730 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46639
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:43577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:10.510666 27770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:10.511000 27770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:10.511049 27770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:10.515220 27770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:10.515300 27770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:10.515376 27770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:10.520944 27770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:43577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:10.522149 27770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:10.523264 27770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:10.533685 27791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:10.533668 27784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:10.533902 27770 server_base.cc:1034] running on GCE node
W20260811 22:21:10.533668 27788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:10.534487 27770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:10.535316 27770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:10.536517 27770 hybrid_clock.cc:648] HybridClock initialized: now 1786486870536466 us; error 35 us; skew 500 ppm
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:10.539561 27770 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:10.540544 27770 webserver.cc:460] Webserver started at http://127.26.125.3:45989/ using document root <none> and password file <none>
I20260811 22:21:10.541141 27770 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:10.541215 27770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:10.541431 27770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:10.543708 27770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "38c56af84e6246c298a255df9aea1873"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "14314011d293a0161416a8251cd5cdbf"
server_key_iv: "f9f08bdd76a0578e6eb97429fd35ed10"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.544328 27770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "38c56af84e6246c298a255df9aea1873"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "14314011d293a0161416a8251cd5cdbf"
server_key_iv: "f9f08bdd76a0578e6eb97429fd35ed10"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.548404 27770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:10.551461 27809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.552807 27770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:10.552953 27770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "38c56af84e6246c298a255df9aea1873"
format_stamp: "Formatted at 2026-08-11 22:21:10 on dist-test-slave-5bn2"
server_key: "14314011d293a0161416a8251cd5cdbf"
server_key_iv: "f9f08bdd76a0578e6eb97429fd35ed10"
server_key_version: "encryptionkey@0"
I20260811 22:21:10.553081 27770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:10.579144 27770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:10.583092 27770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:10.583305 27770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:10.583968 27770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:10.584854 27770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:10.584903 27770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.584946 27770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:10.584960 27770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:10.594172 27770 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:38665
I20260811 22:21:10.594228 28009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:38665 every 8 connection(s)
I20260811 22:21:10.595300 27770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:10.598004 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27770
I20260811 22:21:10.598122 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:10.598428 27124 external_mini_cluster.cc:1428] Setting key 3e1b6a3bf8b98a3c3e3c820f36ffe795
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:10.611214 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:10.597490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:53507 (local address 127.26.125.62:46639)
0811 22:21:10.597658 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:10.597664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:10.598989 (+ 1325us) server_negotiation.cc:407] Connection header received
0811 22:21:10.600128 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:10.600131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:10.600184 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:10.600253 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:10.601708 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.602417 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.604053 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.604401 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.604484 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:10.607525 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:10.607528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:10.607530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:10.607562 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:10.609007 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.609595 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.609598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.609599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.609702 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.610049 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.610052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.610054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.610288 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:10.610373 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:10.610972 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 22:21:10.611085 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:21:10.612257 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:10.597940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:46639 (local address 127.26.125.3:53507)
0811 22:21:10.598760 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:10.598822 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:10.599899 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:10.600471 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:10.600479 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:10.600928 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:10.601549 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:10.601566 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.602549 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.602553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:10.603897 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:10.603911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.604718 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.604722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:10.604963 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:10.605644 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:10.605662 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:10.607337 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:10.609160 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:10.609166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:10.609182 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:10.609448 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:10.609783 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:10.609785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:10.609786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:10.609923 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:10.610399 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:10.610405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:10.610695 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 22:21:10.610888 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 22:21:10.611229 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":160,"threads_started":1}
I20260811 22:21:10.613636 28010 heartbeater.cc:347] Connected to a master server at 127.26.125.62:46639
I20260811 22:21:10.613878 28010 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:10.614374 28010 heartbeater.cc:511] Master 127.26.125.62:46639 requested a full tablet report, sending...
I20260811 22:21:10.615399 27214 ts_manager.cc:194] Registered new tserver with Master: 38c56af84e6246c298a255df9aea1873 (127.26.125.3:38665)
I20260811 22:21:10.615965 27214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:53507
I20260811 22:21:10.625592 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:10.654151 28020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:10.644104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38588 (local address 127.26.125.1:42261)
0811 22:21:10.644473 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:10.644480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:10.644508 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:21:10.644561 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:10.644567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:10.644771 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:10.644899 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:10.645565 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.646350 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.647232 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.647667 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.647772 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:10.650518 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:10.650530 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:10.650547 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:10.650578 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:10.652357 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.652798 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.652804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.652806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.652894 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.653145 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.653147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.653148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.653468 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:10.653667 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:10.653839 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 22:21:10.654005 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":64,"threads_started":1}
W20260811 22:21:10.655181 27393 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38588
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:10.682806 28020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:10.674394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38596 (local address 127.26.125.1:42261)
0811 22:21:10.674588 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:10.674593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:10.674697 (+ 104us) server_negotiation.cc:407] Connection header received
0811 22:21:10.674794 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:10.674797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:10.674922 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:10.675001 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:10.675679 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.676219 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.677019 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:10.677259 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:10.677357 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:10.679952 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:10.679955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:10.679957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:10.679986 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:10.681200 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.681625 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.681628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.681629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.681711 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:10.681969 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:10.681973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:10.681975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:10.682153 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:10.682274 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:10.682522 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 22:21:10.682646 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:21:10.683928 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27326
I20260811 22:21:10.699551 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27549
I20260811 22:21:10.728446 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27770
I20260811 22:21:10.742415 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27162
2026-08-11T22:21:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2460 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[28031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[28031](info): set up 2 sockets
Aug 11 22:21:10 dist-test-slave-5bn2 krb5kdc[28031](info): commencing operation
krb5kdc: starting...
W20260811 22:21:11.949769 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:12 dist-test-slave-5bn2 krb5kdc[28031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:12.090021 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40997
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:12.212030 28051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:12.212378 28051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:12.212440 28051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:12.216625 28051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:12.216713 28051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:12.216740 28051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:12.216763 28051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:12.216784 28051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:12.222491 28051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:12.223862 28051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:12.225060 28051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:12.234519 28061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:12.234534 28062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:12.234613 28069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:12.234907 28051 server_base.cc:1034] running on GCE node
I20260811 22:21:12.235666 28051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:12.236912 28051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:12.238086 28051 hybrid_clock.cc:648] HybridClock initialized: now 1786486872238054 us; error 44 us; skew 500 ppm
Aug 11 22:21:12 dist-test-slave-5bn2 krb5kdc[28031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:12.241027 28051 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:12.242048 28051 webserver.cc:460] Webserver started at http://127.26.125.62:39189/ using document root <none> and password file <none>
I20260811 22:21:12.242679 28051 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:12.242745 28051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:12.242995 28051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:12.245584 28051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "00a148360b154b7aa22a4dcef9f28ca0"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "10dd676f0abffea227dd632b0f0c34d7"
server_key_iv: "09527c91f71b623d3bd97620e58743c8"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.246237 28051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "00a148360b154b7aa22a4dcef9f28ca0"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "10dd676f0abffea227dd632b0f0c34d7"
server_key_iv: "09527c91f71b623d3bd97620e58743c8"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.250762 28051 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 22:21:12.253798 28086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:12.255316 28051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:21:12.255436 28051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "00a148360b154b7aa22a4dcef9f28ca0"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "10dd676f0abffea227dd632b0f0c34d7"
server_key_iv: "09527c91f71b623d3bd97620e58743c8"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.255558 28051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:12.269593 28051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:12.273156 28051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:12.273331 28051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:12.280562 28051 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:40997
I20260811 22:21:12.280756 28175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:40997 every 8 connection(s)
I20260811 22:21:12.281646 28051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:12.284687 28176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:12.288409 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 28051
I20260811 22:21:12.288537 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:12.288812 27124 external_mini_cluster.cc:1428] Setting key 3af74d452095d4880df7490125261efd
I20260811 22:21:12.291626 28176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0: Bootstrap starting.
I20260811 22:21:12.294056 28176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:12.294888 28176 log.cc:824] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:12.297317 28176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0: No bootstrap required, opened a new log
Aug 11 22:21:12 dist-test-slave-5bn2 krb5kdc[28031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:12.301595 28176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a148360b154b7aa22a4dcef9f28ca0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40997 } }
I20260811 22:21:12.301870 28176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:12.301930 28176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a148360b154b7aa22a4dcef9f28ca0, State: Initialized, Role: FOLLOWER
I20260811 22:21:12.302464 28176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a148360b154b7aa22a4dcef9f28ca0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40997 } }
I20260811 22:21:12.302634 28176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:12.302711 28176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:12.302811 28176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:12.304052 28176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a148360b154b7aa22a4dcef9f28ca0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40997 } }
I20260811 22:21:12.304440 28176 leader_election.cc:302] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00a148360b154b7aa22a4dcef9f28ca0; no voters:
I20260811 22:21:12.304783 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:12.290442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.26.125.62:40997)
0811 22:21:12.291007 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:12.291025 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:12.291066 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:21:12.291743 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:12.291771 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:12.292163 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:12.292550 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:12.293326 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.294420 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.295411 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.296021 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.296180 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:12.298638 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:12.298644 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:12.298663 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:12.298693 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:12.301441 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:12.301890 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:12.301906 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:12.301913 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:12.302072 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:12.302362 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:12.302365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:12.302366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:12.302975 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:12.303184 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:12.303409 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:21:12.303729 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":171,"threads_started":1}
I20260811 22:21:12.304813 28176 leader_election.cc:288] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:12.305019 28185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:12.305224 28185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [term 1 LEADER]: Becoming Leader. State: Replica: 00a148360b154b7aa22a4dcef9f28ca0, State: Running, Role: LEADER
I20260811 22:21:12.305562 28185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a148360b154b7aa22a4dcef9f28ca0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40997 } }
I20260811 22:21:12.306222 28176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:12.306891 28185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00a148360b154b7aa22a4dcef9f28ca0. Latest consensus state: current_term: 1 leader_uuid: "00a148360b154b7aa22a4dcef9f28ca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a148360b154b7aa22a4dcef9f28ca0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40997 } } }
I20260811 22:21:12.307004 28185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:12.307343 28187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00a148360b154b7aa22a4dcef9f28ca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a148360b154b7aa22a4dcef9f28ca0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40997 } } }
I20260811 22:21:12.307451 28187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:12.307796 28196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:12.312014 28196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:12.318089 28196 catalog_manager.cc:1347] Generated new cluster ID: b7309f0da9d540dc80be6756a8e2e831
I20260811 22:21:12.318181 28196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:12.330912 28196 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:12.332322 28196 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:12.348178 28196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00a148360b154b7aa22a4dcef9f28ca0: Generated new TSK 0
I20260811 22:21:12.349203 28196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:12.387905 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40997
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:12.505409 28219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:12.505740 28219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:12.505788 28219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:12.510128 28219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:12.510217 28219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:12.510291 28219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:12.515965 28219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:12.517244 28219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:12.518466 28219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:12.529289 28230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:12.529453 28219 server_base.cc:1034] running on GCE node
W20260811 22:21:12.529314 28226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:12.529314 28228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:12.530206 28219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:12.530936 28219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:12.532122 28219 hybrid_clock.cc:648] HybridClock initialized: now 1786486872532091 us; error 50 us; skew 500 ppm
Aug 11 22:21:12 dist-test-slave-5bn2 krb5kdc[28031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:12.538321 28219 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:12.539625 28219 webserver.cc:460] Webserver started at http://127.26.125.1:34329/ using document root <none> and password file <none>
I20260811 22:21:12.540314 28219 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:12.540397 28219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:12.540603 28219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:12.543057 28219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "7745a9b822ec419f8dc077c349a1c19e"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "e9564f6b52d8d6daa78e79223a46a971"
server_key_iv: "9abb10d32da3920efde45f66e72d01c5"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.543742 28219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "7745a9b822ec419f8dc077c349a1c19e"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "e9564f6b52d8d6daa78e79223a46a971"
server_key_iv: "9abb10d32da3920efde45f66e72d01c5"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.548319 28219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:12.551079 28244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:12.552425 28219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:12.552561 28219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "7745a9b822ec419f8dc077c349a1c19e"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "e9564f6b52d8d6daa78e79223a46a971"
server_key_iv: "9abb10d32da3920efde45f66e72d01c5"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.552682 28219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:12.568785 28219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:12.572242 28219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:12.572431 28219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:12.573064 28219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:12.573963 28219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:12.574018 28219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:12.574065 28219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:12.574095 28219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:12.583178 28219 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:36123
I20260811 22:21:12.583209 28445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:36123 every 8 connection(s)
I20260811 22:21:12.584173 28219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:12.585856 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 28219
I20260811 22:21:12.585958 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:12.586236 27124 external_mini_cluster.cc:1428] Setting key c37c654178f2fcf08da45308106c835b
Aug 11 22:21:12 dist-test-slave-5bn2 krb5kdc[28031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:21:12.600255 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:12.586109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:59397 (local address 127.26.125.62:40997)
0811 22:21:12.586281 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:12.586285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:12.587420 (+ 1135us) server_negotiation.cc:407] Connection header received
0811 22:21:12.588989 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:12.588992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:12.589040 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:12.589109 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:12.590904 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.591693 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.592906 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.593262 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.593378 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:12.596299 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:12.596304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:12.596307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:12.596340 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:12.597860 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:12.598417 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:12.598421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:12.598423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:12.598606 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:12.599037 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:12.599039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:12.599040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:12.599248 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:12.599325 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:12.599964 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 22:21:12.600091 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:21:12.601320 28450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:12.586588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40997 (local address 127.26.125.1:59397)
0811 22:21:12.587208 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:12.587274 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:12.588734 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:12.589318 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:12.589328 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:12.589847 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:12.590645 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:12.590664 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.591827 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.591830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:12.592758 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:12.592768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.593499 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.593502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:12.593667 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:12.594274 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:12.594292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:12.596044 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:12.597997 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:12.598003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:12.598021 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:12.598311 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:12.598723 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:12.598725 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:12.598727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:12.598937 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:12.599326 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:12.599332 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:12.599671 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 22:21:12.599889 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 22:21:12.600265 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":166,"threads_started":1}
I20260811 22:21:12.602698 28446 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40997
I20260811 22:21:12.602995 28446 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:12.603530 28446 heartbeater.cc:511] Master 127.26.125.62:40997 requested a full tablet report, sending...
I20260811 22:21:12.605183 28113 ts_manager.cc:194] Registered new tserver with Master: 7745a9b822ec419f8dc077c349a1c19e (127.26.125.1:36123)
I20260811 22:21:12.606422 28113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:59397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:12.617587 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40997
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:12.734184 28455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:12.734479 28455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:12.734542 28455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:12.738585 28455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:12.738658 28455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:12.738741 28455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:12.744298 28455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:12.745433 28455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:12.746491 28455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:12.756520 28473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:12.756476 28467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:12.756577 28455 server_base.cc:1034] running on GCE node
W20260811 22:21:12.756475 28468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:12.757266 28455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:12.757965 28455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:12.759128 28455 hybrid_clock.cc:648] HybridClock initialized: now 1786486872759057 us; error 83 us; skew 500 ppm
Aug 11 22:21:12 dist-test-slave-5bn2 krb5kdc[28031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:12.762149 28455 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:12.763298 28455 webserver.cc:460] Webserver started at http://127.26.125.2:35787/ using document root <none> and password file <none>
I20260811 22:21:12.763903 28455 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:12.763972 28455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:12.764168 28455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:12.766379 28455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "252ef9b9229c44929bab0c5ce3f5c2b5"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "2df409bbcdb56522b713cd2ba813f242"
server_key_iv: "5f2379abee08facadfd09bcbb71443e1"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.767007 28455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "252ef9b9229c44929bab0c5ce3f5c2b5"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "2df409bbcdb56522b713cd2ba813f242"
server_key_iv: "5f2379abee08facadfd09bcbb71443e1"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.771114 28455 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:21:12.773672 28488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:12.775032 28455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:21:12.775147 28455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "252ef9b9229c44929bab0c5ce3f5c2b5"
format_stamp: "Formatted at 2026-08-11 22:21:12 on dist-test-slave-5bn2"
server_key: "2df409bbcdb56522b713cd2ba813f242"
server_key_iv: "5f2379abee08facadfd09bcbb71443e1"
server_key_version: "encryptionkey@0"
I20260811 22:21:12.775252 28455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:12.789598 28455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:12.793668 28455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:12.793895 28455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:12.794579 28455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:12.795513 28455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:12.795562 28455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:12.795606 28455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:12.795634 28455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:12.804483 28455 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:45835
I20260811 22:21:12.804512 28685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:45835 every 8 connection(s)
I20260811 22:21:12.805450 28455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:12.816556 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 28455
I20260811 22:21:12.816665 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:12.816879 27124 external_mini_cluster.cc:1428] Setting key 07de2391e79f4f089d39e7018239d868
Aug 11 22:21:12 dist-test-slave-5bn2 krb5kdc[28031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:12.826694 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:12.807769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:44837 (local address 127.26.125.62:40997)
0811 22:21:12.807887 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:12.807892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:12.809257 (+ 1365us) server_negotiation.cc:407] Connection header received
0811 22:21:12.810546 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:12.810550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:12.810608 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:12.810704 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:12.812457 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.813265 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.818475 (+ 5210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.818725 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.818811 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:12.822177 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:12.822181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:12.822184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:12.822213 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:12.823921 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:12.824601 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:12.824605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:12.824607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:12.824711 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:12.825164 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:12.825168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:12.825170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:12.825438 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:12.825544 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:12.826349 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 22:21:12.826501 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 22:21:12.827903 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:12.808441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40997 (local address 127.26.125.2:44837)
0811 22:21:12.809061 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:12.809128 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:12.810263 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:12.810938 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:12.810948 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:12.811512 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:12.812277 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:12.812301 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.813429 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.813433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:12.817462 (+ 4029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:12.817480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:12.818949 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:12.818955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:12.819199 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:12.819961 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:12.819995 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:12.821922 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:12.824052 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:12.824060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:12.824084 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:12.824472 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:12.824809 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:12.824812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:12.824814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:12.825033 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:12.825620 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:12.825629 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:12.826016 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 22:21:12.826250 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:21:12.826676 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:12.829288 28686 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40997
I20260811 22:21:12.829560 28686 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:12.830224 28686 heartbeater.cc:511] Master 127.26.125.62:40997 requested a full tablet report, sending...
I20260811 22:21:12.832711 28113 ts_manager.cc:194] Registered new tserver with Master: 252ef9b9229c44929bab0c5ce3f5c2b5 (127.26.125.2:45835)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:21:12.833410 28113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:44837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:12.854573 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40997
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:12.977473 28693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:12.977758 28693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:12.977821 28693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:12.981992 28693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:12.982067 28693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:12.982151 28693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:12.987495 28693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40997
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:12.988638 28693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:12.989734 28693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:13.000594 28702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:13.000691 28693 server_base.cc:1034] running on GCE node
W20260811 22:21:13.000645 28704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:13.000594 28701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:13.001335 28693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:13.002071 28693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:13.003232 28693 hybrid_clock.cc:648] HybridClock initialized: now 1786486873003206 us; error 36 us; skew 500 ppm
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486873, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:13.006240 28693 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:13.007603 28693 webserver.cc:460] Webserver started at http://127.26.125.3:38935/ using document root <none> and password file <none>
I20260811 22:21:13.008404 28693 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:13.008509 28693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:13.008780 28693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:13.011857 28693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "fa210ae1c4ea4195ab6c663431891789"
format_stamp: "Formatted at 2026-08-11 22:21:13 on dist-test-slave-5bn2"
server_key: "b7ccc506884dab2d29419d78818a6f89"
server_key_iv: "970595190db6b1bbfb26ff2e574fa514"
server_key_version: "encryptionkey@0"
I20260811 22:21:13.012610 28693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "fa210ae1c4ea4195ab6c663431891789"
format_stamp: "Formatted at 2026-08-11 22:21:13 on dist-test-slave-5bn2"
server_key: "b7ccc506884dab2d29419d78818a6f89"
server_key_iv: "970595190db6b1bbfb26ff2e574fa514"
server_key_version: "encryptionkey@0"
I20260811 22:21:13.017647 28693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:21:13.020347 28723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:13.021656 28693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:21:13.021777 28693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "fa210ae1c4ea4195ab6c663431891789"
format_stamp: "Formatted at 2026-08-11 22:21:13 on dist-test-slave-5bn2"
server_key: "b7ccc506884dab2d29419d78818a6f89"
server_key_iv: "970595190db6b1bbfb26ff2e574fa514"
server_key_version: "encryptionkey@0"
I20260811 22:21:13.021881 28693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:13.055102 28693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:13.058660 28693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:13.058918 28693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:13.059594 28693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:13.060523 28693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:13.060572 28693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:13.060616 28693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:13.060642 28693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:13.069396 28693 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:46555
I20260811 22:21:13.069471 28917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:46555 every 8 connection(s)
I20260811 22:21:13.070353 28693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:13.074308 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 28693
I20260811 22:21:13.074481 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:13.074817 27124 external_mini_cluster.cc:1428] Setting key 9de6ef2ca2678107036bb752aba045a3
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486873, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:13.084945 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.072277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:33115 (local address 127.26.125.62:40997)
0811 22:21:13.072407 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.072410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.073392 (+ 982us) server_negotiation.cc:407] Connection header received
0811 22:21:13.074339 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.074343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.074406 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.074492 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:13.076346 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.077102 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.078382 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.078614 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.078724 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.081462 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:13.081466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:13.081468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:13.081502 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:13.083008 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.083537 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.083540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.083541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.083622 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.083972 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.083974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.083975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.084153 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:13.084224 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.084712 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.084817 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:21:13.085901 28920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.072710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40997 (local address 127.26.125.3:33115)
0811 22:21:13.073227 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:13.073276 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:13.074116 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:13.074811 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:13.074821 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:13.075377 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:13.076137 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.076159 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.077247 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.077250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.078253 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.078265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.078777 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.078782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.079005 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.079626 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:13.079644 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:13.081252 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:13.083117 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:13.083123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:13.083138 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:13.083431 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:13.083728 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:13.083731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:13.083732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:13.083895 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:13.084228 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:13.084233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:13.084505 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 22:21:13.084679 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 22:21:13.084982 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":146,"threads_started":1}
I20260811 22:21:13.087209 28918 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40997
I20260811 22:21:13.087487 28918 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:13.088084 28918 heartbeater.cc:511] Master 127.26.125.62:40997 requested a full tablet report, sending...
I20260811 22:21:13.089164 28113 ts_manager.cc:194] Registered new tserver with Master: fa210ae1c4ea4195ab6c663431891789 (127.26.125.3:46555)
I20260811 22:21:13.089725 28113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:33115
I20260811 22:21:13.101051 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:21:13.112314 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.104920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40284 (local address 127.26.125.62:40997)
0811 22:21:13.105041 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.105045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.105145 (+ 100us) server_negotiation.cc:407] Connection header received
0811 22:21:13.105257 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.105259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.105299 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.105381 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:13.106077 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.106663 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.107621 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.107878 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.107979 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.109451 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:13.109454 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:13.109456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:13.109482 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:13.110817 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.111271 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.111274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.111275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.111357 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.111555 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.111556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.111558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.111789 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:13.111892 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.112079 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.112185 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:21:13.117720 28113 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:13.120898 28113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:13.136193 28943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.130321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.26.125.2:45835)
0811 22:21:13.130799 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.130817 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.131046 (+ 229us) server_negotiation.cc:407] Connection header received
0811 22:21:13.131148 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.131154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.131394 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.131994 (+ 600us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:13.132148 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.133072 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.134601 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.135439 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.135547 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.135711 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.135904 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.136021 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":141,"threads_started":1}
I20260811 22:21:13.136184 28945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.130452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:36123 (local address 127.0.0.1:53480)
0811 22:21:13.131506 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:13.131520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:13.131618 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:13.132162 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:13.132165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:13.132183 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:13.132262 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.132267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.133612 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.133615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.134765 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.134774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.135815 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.135818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.135913 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.135946 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:21:13.136006 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 22:21:13.136078 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":49,"thread_start_us":116,"threads_started":1}
I20260811 22:21:13.136184 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.130138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:45835 (local address 127.0.0.1:42732)
0811 22:21:13.130539 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:13.130601 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:13.130776 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:13.131616 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:13.131623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:13.131649 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:13.131821 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.131829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.133185 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.133188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.134462 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.134474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.135572 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.135576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.135700 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.135743 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:21:13.135858 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:21:13.135997 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260811 22:21:13.136534 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.130958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:46555 (local address 127.0.0.1:45706)
0811 22:21:13.131856 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:13.131869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:13.131946 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:13.132615 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:13.132618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:13.132637 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:13.132715 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.132720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.133922 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.133925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.134955 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.134965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.136136 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.136140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.136224 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.136247 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:21:13.136315 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:21:13.136392 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"thread_start_us":135,"threads_started":1}
I20260811 22:21:13.136662 28948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.131051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.26.125.1:36123)
0811 22:21:13.131318 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.131324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.131730 (+ 406us) server_negotiation.cc:407] Connection header received
0811 22:21:13.131815 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.131821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.132030 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.132184 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:13.132358 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.133490 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.134938 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.135505 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.135608 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.135753 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.136417 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.136526 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":100,"threads_started":1}
I20260811 22:21:13.136931 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.131536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45706 (local address 127.26.125.3:46555)
0811 22:21:13.132185 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.132196 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.132219 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:21:13.132292 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.132297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.132483 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.132650 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:13.132807 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.133797 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.135083 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.135843 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.135943 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.136069 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.136703 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.136811 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":144,"threads_started":1}
I20260811 22:21:13.139963 28327 tablet_service.cc:1467] Processing CreateTablet for tablet 061cadf854614d31b3209b366a1fde01 (DEFAULT_TABLE table=test-table [id=bdce3889b1564089a729759027c875c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:13.140539 28799 tablet_service.cc:1467] Processing CreateTablet for tablet 061cadf854614d31b3209b366a1fde01 (DEFAULT_TABLE table=test-table [id=bdce3889b1564089a729759027c875c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:13.140539 28575 tablet_service.cc:1467] Processing CreateTablet for tablet 061cadf854614d31b3209b366a1fde01 (DEFAULT_TABLE table=test-table [id=bdce3889b1564089a729759027c875c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:13.141043 28327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061cadf854614d31b3209b366a1fde01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:13.141674 28799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061cadf854614d31b3209b366a1fde01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:13.141674 28575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061cadf854614d31b3209b366a1fde01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:13.147634 28952 tablet_bootstrap.cc:492] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e: Bootstrap starting.
I20260811 22:21:13.148947 28956 tablet_bootstrap.cc:492] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5: Bootstrap starting.
I20260811 22:21:13.150511 28952 tablet_bootstrap.cc:654] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:13.151188 28955 tablet_bootstrap.cc:492] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789: Bootstrap starting.
I20260811 22:21:13.151496 28952 log.cc:824] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:13.151587 28956 tablet_bootstrap.cc:654] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:13.152366 28956 log.cc:824] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:13.153172 28955 tablet_bootstrap.cc:654] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:13.154062 28952 tablet_bootstrap.cc:492] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e: No bootstrap required, opened a new log
I20260811 22:21:13.154212 28955 log.cc:824] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:13.154260 28952 ts_tablet_manager.cc:1397] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:21:13.156193 28955 tablet_bootstrap.cc:492] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789: No bootstrap required, opened a new log
I20260811 22:21:13.156366 28955 ts_tablet_manager.cc:1397] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:21:13.157876 28952 raft_consensus.cc:348] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.158114 28952 raft_consensus.cc:374] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:13.158190 28952 raft_consensus.cc:729] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7745a9b822ec419f8dc077c349a1c19e, State: Initialized, Role: FOLLOWER
I20260811 22:21:13.158682 28952 consensus_queue.cc:260] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.159435 28446 heartbeater.cc:503] Master 127.26.125.62:40997 was elected leader, sending a full tablet report...
I20260811 22:21:13.159484 28955 raft_consensus.cc:348] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.159729 28955 raft_consensus.cc:374] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:13.159782 28955 raft_consensus.cc:729] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa210ae1c4ea4195ab6c663431891789, State: Initialized, Role: FOLLOWER
I20260811 22:21:13.159862 28952 ts_tablet_manager.cc:1428] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:21:13.160245 28955 consensus_queue.cc:260] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.161203 28955 ts_tablet_manager.cc:1428] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:21:13.161556 28918 heartbeater.cc:503] Master 127.26.125.62:40997 was elected leader, sending a full tablet report...
I20260811 22:21:13.162031 28956 tablet_bootstrap.cc:492] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5: No bootstrap required, opened a new log
I20260811 22:21:13.162245 28956 ts_tablet_manager.cc:1397] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260811 22:21:13.165772 28956 raft_consensus.cc:348] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.166023 28956 raft_consensus.cc:374] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:13.166065 28956 raft_consensus.cc:729] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 252ef9b9229c44929bab0c5ce3f5c2b5, State: Initialized, Role: FOLLOWER
I20260811 22:21:13.166436 28956 consensus_queue.cc:260] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.167081 28686 heartbeater.cc:503] Master 127.26.125.62:40997 was elected leader, sending a full tablet report...
I20260811 22:21:13.167846 28956 ts_tablet_manager.cc:1428] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:21:13.262197 28966 raft_consensus.cc:482] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:13.262414 28966 raft_consensus.cc:504] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.263607 28966 leader_election.cc:288] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 252ef9b9229c44929bab0c5ce3f5c2b5 (127.26.125.2:45835), fa210ae1c4ea4195ab6c663431891789 (127.26.125.3:46555)
I20260811 22:21:13.267788 28943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.264008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:55391 (local address 127.26.125.2:45835)
0811 22:21:13.264149 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.264152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.264306 (+ 154us) server_negotiation.cc:407] Connection header received
0811 22:21:13.264524 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.264528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.264594 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.264715 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:13.265066 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.265604 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.266747 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.267188 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.267311 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.267377 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.267554 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.267635 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:21:13.267807 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.264467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:42585 (local address 127.26.125.3:46555)
0811 22:21:13.264608 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.264612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.264770 (+ 158us) server_negotiation.cc:407] Connection header received
0811 22:21:13.264903 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.264906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.264952 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.265012 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:13.265270 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.265759 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.266769 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.267253 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.267362 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.267416 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.267590 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.267667 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:21:13.267861 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.264321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:46555 (local address 127.26.125.1:42585)
0811 22:21:13.264669 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:13.264680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:13.264804 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:13.265042 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:13.265044 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:13.265061 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:13.265167 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.265172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.265870 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.265874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.266646 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.266655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.267371 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.267374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.267482 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.267501 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:21:13.267566 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:21:13.267668 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":25,"spinlock_wait_cycles":4864,"thread_start_us":106,"threads_started":1}
I20260811 22:21:13.267874 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.263892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:45835 (local address 127.26.125.1:55391)
0811 22:21:13.264207 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:13.264221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:13.264345 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:13.264736 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:13.264739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:13.264795 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:13.264936 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.264943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.265745 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.265749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.266607 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:13.266617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.267311 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.267316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:13.267425 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.267451 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:21:13.267530 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:21:13.267632 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":1,"spinlock_wait_cycles":41984,"thread_start_us":114,"threads_started":1}
I20260811 22:21:13.268604 28608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061cadf854614d31b3209b366a1fde01" candidate_uuid: "7745a9b822ec419f8dc077c349a1c19e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" is_pre_election: true
I20260811 22:21:13.268612 28838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061cadf854614d31b3209b366a1fde01" candidate_uuid: "7745a9b822ec419f8dc077c349a1c19e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa210ae1c4ea4195ab6c663431891789" is_pre_election: true
I20260811 22:21:13.268906 28608 raft_consensus.cc:2454] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7745a9b822ec419f8dc077c349a1c19e in term 0.
I20260811 22:21:13.268908 28838 raft_consensus.cc:2454] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7745a9b822ec419f8dc077c349a1c19e in term 0.
I20260811 22:21:13.269376 28245 leader_election.cc:302] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7745a9b822ec419f8dc077c349a1c19e, fa210ae1c4ea4195ab6c663431891789; no voters:
I20260811 22:21:13.269649 28966 raft_consensus.cc:2781] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:13.269739 28966 raft_consensus.cc:482] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:13.269775 28966 raft_consensus.cc:3037] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:13.271052 28966 raft_consensus.cc:504] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.271441 28966 leader_election.cc:288] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [CANDIDATE]: Term 1 election: Requested vote from peers 252ef9b9229c44929bab0c5ce3f5c2b5 (127.26.125.2:45835), fa210ae1c4ea4195ab6c663431891789 (127.26.125.3:46555)
I20260811 22:21:13.271834 28608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061cadf854614d31b3209b366a1fde01" candidate_uuid: "7745a9b822ec419f8dc077c349a1c19e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5"
I20260811 22:21:13.271869 28838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061cadf854614d31b3209b366a1fde01" candidate_uuid: "7745a9b822ec419f8dc077c349a1c19e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa210ae1c4ea4195ab6c663431891789"
I20260811 22:21:13.271955 28608 raft_consensus.cc:3037] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:13.271981 28838 raft_consensus.cc:3037] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:13.273037 28608 raft_consensus.cc:2454] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7745a9b822ec419f8dc077c349a1c19e in term 1.
I20260811 22:21:13.273085 28838 raft_consensus.cc:2454] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7745a9b822ec419f8dc077c349a1c19e in term 1.
I20260811 22:21:13.273361 28245 leader_election.cc:302] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 252ef9b9229c44929bab0c5ce3f5c2b5, 7745a9b822ec419f8dc077c349a1c19e; no voters:
I20260811 22:21:13.273569 28966 raft_consensus.cc:2781] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:13.273805 28966 raft_consensus.cc:686] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [term 1 LEADER]: Becoming Leader. State: Replica: 7745a9b822ec419f8dc077c349a1c19e, State: Running, Role: LEADER
I20260811 22:21:13.274169 28966 consensus_queue.cc:237] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } }
I20260811 22:21:13.277701 28112 catalog_manager.cc:5495] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7745a9b822ec419f8dc077c349a1c19e (127.26.125.1). New cstate: current_term: 1 leader_uuid: "7745a9b822ec419f8dc077c349a1c19e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "252ef9b9229c44929bab0c5ce3f5c2b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa210ae1c4ea4195ab6c663431891789" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7745a9b822ec419f8dc077c349a1c19e" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36123 } health_report { overall_health: HEALTHY } } }
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:13.300458 28948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.290712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.26.125.1:36123)
0811 22:21:13.290903 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.290908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.291093 (+ 185us) server_negotiation.cc:407] Connection header received
0811 22:21:13.291202 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.291204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.291251 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.291327 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:13.292163 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.292693 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.293756 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.293987 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.294075 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.296550 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:13.296565 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:13.296582 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:13.296612 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:13.298452 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.298929 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.298937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.298939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.299052 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.299344 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.299347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.299348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.299716 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:13.299930 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.300099 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.300220 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:21:13.307307 28688 tablet.cc:2369] T 061cadf854614d31b3209b366a1fde01 P 252ef9b9229c44929bab0c5ce3f5c2b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486873, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260811 22:21:13.321419 28919 tablet.cc:2369] T 061cadf854614d31b3209b366a1fde01 P fa210ae1c4ea4195ab6c663431891789: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486873, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:13.330811 28948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.322440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.26.125.1:36123)
0811 22:21:13.322608 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.322613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.322774 (+ 161us) server_negotiation.cc:407] Connection header received
0811 22:21:13.322989 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.322993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.323046 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.323117 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:13.323831 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.324349 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.325178 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.325446 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.325538 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.328091 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:13.328094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:13.328096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:13.328123 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:13.329314 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.329744 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.329747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.329748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.329827 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.330020 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.330022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.330023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.330197 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:13.330294 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.330557 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.330657 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 22:21:13.336652 28449 tablet.cc:2369] T 061cadf854614d31b3209b366a1fde01 P 7745a9b822ec419f8dc077c349a1c19e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:13.359318 28948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:13.350541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.26.125.1:36123)
0811 22:21:13.350704 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:13.350709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:13.350919 (+ 210us) server_negotiation.cc:407] Connection header received
0811 22:21:13.351105 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:13.351108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:13.351155 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:13.351223 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:13.351966 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.352470 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.353356 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:13.353585 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:13.353678 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:13.356111 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:13.356115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:13.356116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:13.356145 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:13.357665 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.358073 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.358077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.358079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.358160 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:13.358410 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:13.358413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:13.358415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:13.358609 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:13.358754 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:13.359011 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 22:21:13.359131 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 22:21:13.360841 28294 tablet_service.cc:2908] Rejecting scan request for tablet 061cadf854614d31b3209b366a1fde01: Uninitialized: clean time has not yet been initialized
I20260811 22:21:13.362931 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 28219
I20260811 22:21:13.375942 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 28455
I20260811 22:21:13.397094 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 28693
I20260811 22:21:13.408737 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 28051
2026-08-11T22:21:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2666 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28994](info): set up 2 sockets
Aug 11 22:21:13 dist-test-slave-5bn2 krb5kdc[28994](info): commencing operation
krb5kdc: starting...
W20260811 22:21:14.615038 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:14 dist-test-slave-5bn2 krb5kdc[28994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:14.745533 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44593
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:44593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:14.864091 29013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:14.864432 29013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:14.864513 29013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:14.868707 29013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:14.868785 29013 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:21:14.868808 29013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:14.868831 29013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:14.868853 29013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:14.868881 29013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:14.874488 29013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:44593
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:14.875697 29013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:14.876807 29013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:14.886087 29026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:14.886143 29029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:14.886140 29024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:14.886190 29013 server_base.cc:1034] running on GCE node
I20260811 22:21:14.887108 29013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:14.888393 29013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:14.889588 29013 hybrid_clock.cc:648] HybridClock initialized: now 1786486874889567 us; error 34 us; skew 500 ppm
Aug 11 22:21:14 dist-test-slave-5bn2 krb5kdc[28994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486874, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:14.892714 29013 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:14.893714 29013 webserver.cc:460] Webserver started at http://127.26.125.62:38049/ using document root <none> and password file <none>
I20260811 22:21:14.894311 29013 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:14.894387 29013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:14.894613 29013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:14.896898 29013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "07c87f321cf34deeabcdf69ce325e7a9"
format_stamp: "Formatted at 2026-08-11 22:21:14 on dist-test-slave-5bn2"
server_key: "de9e64c1ed25505515292449d8ec98bd"
server_key_iv: "a6fea4d4bb9bfa4c6e17eec2f3bce6b9"
server_key_version: "encryptionkey@0"
I20260811 22:21:14.897521 29013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "07c87f321cf34deeabcdf69ce325e7a9"
format_stamp: "Formatted at 2026-08-11 22:21:14 on dist-test-slave-5bn2"
server_key: "de9e64c1ed25505515292449d8ec98bd"
server_key_iv: "a6fea4d4bb9bfa4c6e17eec2f3bce6b9"
server_key_version: "encryptionkey@0"
I20260811 22:21:14.901680 29013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:14.904567 29040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:14.905937 29013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:14.906085 29013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "07c87f321cf34deeabcdf69ce325e7a9"
format_stamp: "Formatted at 2026-08-11 22:21:14 on dist-test-slave-5bn2"
server_key: "de9e64c1ed25505515292449d8ec98bd"
server_key_iv: "a6fea4d4bb9bfa4c6e17eec2f3bce6b9"
server_key_version: "encryptionkey@0"
I20260811 22:21:14.906196 29013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:14.925092 29013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:14.928640 29013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:14.928838 29013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:14.937168 29013 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:44593
I20260811 22:21:14.937214 29138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:44593 every 8 connection(s)
I20260811 22:21:14.938179 29013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:14.941151 29139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:14.943358 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 29013
I20260811 22:21:14.943486 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:14.943723 27124 external_mini_cluster.cc:1428] Setting key f4b44eebc70f7a7f3f030e63f2c6b297
I20260811 22:21:14.947491 29139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9: Bootstrap starting.
I20260811 22:21:14.949843 29139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:14.950608 29139 log.cc:824] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:21:14 dist-test-slave-5bn2 krb5kdc[28994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:14.952823 29139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9: No bootstrap required, opened a new log
I20260811 22:21:14.956494 29139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c87f321cf34deeabcdf69ce325e7a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44593 } }
I20260811 22:21:14.956816 29139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:14.956892 29139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c87f321cf34deeabcdf69ce325e7a9, State: Initialized, Role: FOLLOWER
I20260811 22:21:14.957377 29139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c87f321cf34deeabcdf69ce325e7a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44593 } }
I20260811 22:21:14.957511 29139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:14.957576 29139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:14.957674 29139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:14.957909 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:14.945248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.26.125.62:44593)
0811 22:21:14.945713 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:14.945750 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:14.945800 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:21:14.946465 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:14.946493 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:14.946844 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:14.947221 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:14.948020 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:14.949001 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:14.949845 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:14.950393 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:14.950555 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:14.952999 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:14.953004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:14.953021 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:14.953051 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:14.954779 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:14.955246 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:14.955251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:14.955257 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:14.955391 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:14.955670 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:14.955673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:14.955674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:14.956119 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:14.956320 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:14.956619 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 22:21:14.956969 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":151,"threads_started":1}
I20260811 22:21:14.958683 29139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c87f321cf34deeabcdf69ce325e7a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44593 } }
I20260811 22:21:14.959069 29139 leader_election.cc:302] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07c87f321cf34deeabcdf69ce325e7a9; no voters:
I20260811 22:21:14.959414 29139 leader_election.cc:288] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:14.959508 29148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:14.959694 29148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [term 1 LEADER]: Becoming Leader. State: Replica: 07c87f321cf34deeabcdf69ce325e7a9, State: Running, Role: LEADER
I20260811 22:21:14.960083 29148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c87f321cf34deeabcdf69ce325e7a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44593 } }
I20260811 22:21:14.960870 29139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:14.962095 29152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07c87f321cf34deeabcdf69ce325e7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c87f321cf34deeabcdf69ce325e7a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44593 } } }
I20260811 22:21:14.962109 29151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07c87f321cf34deeabcdf69ce325e7a9. Latest consensus state: current_term: 1 leader_uuid: "07c87f321cf34deeabcdf69ce325e7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c87f321cf34deeabcdf69ce325e7a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44593 } } }
I20260811 22:21:14.962368 29152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:14.962388 29151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9 [sys.catalog]: This master's current role is: LEADER
W20260811 22:21:14.966193 29183 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:14.966286 29183 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:14.966526 29168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:14.967329 29168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:14.974014 29168 catalog_manager.cc:1347] Generated new cluster ID: 01a9db2d890a40dfa5487edd05203cf1
I20260811 22:21:14.974098 29168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:14.988274 29168 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:14.989683 29168 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:15.000916 29168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9: Generated new TSK 0
I20260811 22:21:15.001884 29168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:15.037959 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44593
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:15.155807 29191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:15.156169 29191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:15.156246 29191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:15.160364 29191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:15.160437 29191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:15.160492 29191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:15.160557 29191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:15.166306 29191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:44593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:15.167520 29191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:15.168629 29191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:15.178691 29198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:15.178663 29202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:15.178665 29197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:15.178740 29191 server_base.cc:1034] running on GCE node
I20260811 22:21:15.179437 29191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:15.180172 29191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:15.181360 29191 hybrid_clock.cc:648] HybridClock initialized: now 1786486875181300 us; error 74 us; skew 500 ppm
Aug 11 22:21:15 dist-test-slave-5bn2 krb5kdc[28994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:15.184338 29191 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:15.185343 29191 webserver.cc:460] Webserver started at http://127.26.125.1:36199/ using document root <none> and password file <none>
I20260811 22:21:15.185945 29191 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:15.186017 29191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:15.186228 29191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:15.188413 29191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "490733fb2ae24022bc1ae09c0b7a867a"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "9616be769dffc8ebb9cb5995f21f0803"
server_key_iv: "2c719272b96638e69c5a07196c7c728e"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.188999 29191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "490733fb2ae24022bc1ae09c0b7a867a"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "9616be769dffc8ebb9cb5995f21f0803"
server_key_iv: "2c719272b96638e69c5a07196c7c728e"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.193001 29191 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:15.195626 29216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:15.196875 29191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:21:15.196993 29191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "490733fb2ae24022bc1ae09c0b7a867a"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "9616be769dffc8ebb9cb5995f21f0803"
server_key_iv: "2c719272b96638e69c5a07196c7c728e"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.197098 29191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:15.209824 29191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:15.213234 29191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:15.213421 29191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:15.215027 29191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:15.215083 29191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:15.215142 29191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:15.215171 29191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:21:15 dist-test-slave-5bn2 krb5kdc[28994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:15.225724 29191 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:42159
I20260811 22:21:15.225792 29416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:42159 every 8 connection(s)
I20260811 22:21:15.226781 29191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:15.228597 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.215847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:45793 (local address 127.26.125.62:44593)
0811 22:21:15.215985 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.215989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.217116 (+ 1127us) server_negotiation.cc:407] Connection header received
0811 22:21:15.218171 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.218175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.218243 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.218345 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:15.219687 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.220241 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.221407 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.221668 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.221774 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.224681 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:15.224685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:15.224686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:15.224714 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:15.226291 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.226900 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.226904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.226906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.227000 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.227369 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.227373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.227375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.227632 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:15.227729 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.228260 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.228411 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:21:15.229377 29228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.216431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44593 (local address 127.26.125.1:45793)
0811 22:21:15.216947 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.216996 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.217883 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.218559 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.218568 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.218995 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:15.219530 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.219546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.220370 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.220373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.221281 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.221291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.221829 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.221833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.222009 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.222669 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:15.222691 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:15.224504 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:15.226406 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.226411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.226427 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.226760 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.227084 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.227087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.227089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.227280 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.227733 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:15.227740 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:15.227995 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.228195 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.228526 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":137,"threads_started":1}
I20260811 22:21:15.230703 29417 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44593
I20260811 22:21:15.230839 29417 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:15.231374 29417 heartbeater.cc:511] Master 127.26.125.62:44593 requested a full tablet report, sending...
I20260811 22:21:15.233044 29068 ts_manager.cc:194] Registered new tserver with Master: 490733fb2ae24022bc1ae09c0b7a867a (127.26.125.1:42159)
I20260811 22:21:15.234190 29068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:45793
I20260811 22:21:15.235821 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 29191
I20260811 22:21:15.235927 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:15.236193 27124 external_mini_cluster.cc:1428] Setting key bc3c945cb7d5e2c193e173bfd8352229
I20260811 22:21:15.242022 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.232776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44593 (local address 127.26.125.1:56293)
0811 22:21:15.234922 (+ 2146us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.234935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.235040 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.235253 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.235255 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.235461 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:15.235589 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.235597 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.236666 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.236669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.237473 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.237491 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.237927 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.237931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.238033 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.238511 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:15.238526 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:15.238956 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:15.240597 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.240600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.240603 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.240804 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.241056 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.241059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.241061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.241146 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.241545 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:15.241548 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:15.241638 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.241729 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.241856 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2028,"thread_start_us":92,"threads_started":1}
I20260811 22:21:15.242079 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.232860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:56293 (local address 127.26.125.62:44593)
0811 22:21:15.232987 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.232991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.235001 (+ 2010us) server_negotiation.cc:407] Connection header received
0811 22:21:15.235118 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.235121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.235163 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.235224 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:15.235722 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.236524 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.237605 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.237818 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.237897 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.239104 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:15.239107 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:15.239109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:15.239131 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:15.240472 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.240893 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.240896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.240897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.240972 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.241227 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.241230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.241232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.241442 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:15.241537 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.241798 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.241921 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:15.264248 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44593
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:15.379626 29431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:15.379921 29431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:15.379988 29431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:15.384099 29431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:15.384189 29431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:15.384246 29431 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:15.384306 29431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:15.389941 29431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:44593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:15.391261 29431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:15.392405 29431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:15.402335 29438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:15.402335 29445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:15.402335 29439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:15.402386 29431 server_base.cc:1034] running on GCE node
I20260811 22:21:15.403043 29431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:15.403677 29431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:15.404841 29431 hybrid_clock.cc:648] HybridClock initialized: now 1786486875404819 us; error 30 us; skew 500 ppm
Aug 11 22:21:15 dist-test-slave-5bn2 krb5kdc[28994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:15.407603 29431 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:15.408524 29431 webserver.cc:460] Webserver started at http://127.26.125.2:37525/ using document root <none> and password file <none>
I20260811 22:21:15.409094 29431 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:15.409163 29431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:15.409374 29431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:15.411580 29431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "40a89822a4574a09ac03f3f7bf87b064"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "7d7198b025e49cb7c8bf7ab9ad6acfb1"
server_key_iv: "6a28d908ff262d83eb014038d753d2d9"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.412145 29431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "40a89822a4574a09ac03f3f7bf87b064"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "7d7198b025e49cb7c8bf7ab9ad6acfb1"
server_key_iv: "6a28d908ff262d83eb014038d753d2d9"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.416344 29431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:15.418918 29455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:15.420215 29431 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:21:15.420341 29431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "40a89822a4574a09ac03f3f7bf87b064"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "7d7198b025e49cb7c8bf7ab9ad6acfb1"
server_key_iv: "6a28d908ff262d83eb014038d753d2d9"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.420451 29431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:15.433581 29431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:15.436926 29431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:15.437125 29431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:15.438651 29431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:15.438707 29431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:15.438763 29431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:15.438789 29431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:21:15 dist-test-slave-5bn2 krb5kdc[28994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:15.449031 29431 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:41461
I20260811 22:21:15.449427 29676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:41461 every 8 connection(s)
I20260811 22:21:15.450143 29431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:15.451521 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 29431
I20260811 22:21:15.451632 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:15.451934 27124 external_mini_cluster.cc:1428] Setting key 575bb29a0fceb69de29550938740e59b
I20260811 22:21:15.452669 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.439538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:50535 (local address 127.26.125.62:44593)
0811 22:21:15.439660 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.439664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.440818 (+ 1154us) server_negotiation.cc:407] Connection header received
0811 22:21:15.442041 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.442045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.442108 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.442191 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:15.443598 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.444194 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.445405 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.445634 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.445723 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.448885 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:15.448890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:15.448892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:15.448925 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:15.450366 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.450952 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.450956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.450957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.451040 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.451384 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.451387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.451388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.451652 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:15.451754 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.452379 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.452513 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:21:15.453645 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.440141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44593 (local address 127.26.125.2:50535)
0811 22:21:15.440660 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.440706 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.441760 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.442390 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.442401 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.442907 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:15.443446 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.443461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.444308 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.444312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.445281 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.445292 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.445790 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.445794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.445956 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.447147 (+ 1191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:15.447164 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:15.448674 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:15.450483 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.450490 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.450507 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.450840 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.451125 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.451128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.451130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.451268 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.451759 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:15.451765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:15.452106 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.452314 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.452639 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":137,"threads_started":1}
I20260811 22:21:15.454823 29677 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44593
I20260811 22:21:15.455003 29677 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:15.455538 29677 heartbeater.cc:511] Master 127.26.125.62:44593 requested a full tablet report, sending...
I20260811 22:21:15.456727 29070 ts_manager.cc:194] Registered new tserver with Master: 40a89822a4574a09ac03f3f7bf87b064 (127.26.125.2:41461)
I20260811 22:21:15.457373 29070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:50535
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:15.465863 29691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.457646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44593 (local address 127.26.125.2:37643)
0811 22:21:15.459019 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.459032 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.459128 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.459345 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.459349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.459562 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:15.459654 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.459659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.460379 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.460383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.460967 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.460976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.461385 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.461390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.461487 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.462026 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:15.462041 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:15.462389 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:15.464271 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.464274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.464276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.464486 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.464746 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.464749 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.464751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.464836 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.465304 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:15.465308 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:15.465409 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.465505 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.465666 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1296,"thread_start_us":109,"threads_started":1}
I20260811 22:21:15.465880 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.457756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:37643 (local address 127.26.125.62:44593)
0811 22:21:15.457868 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.457871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.459104 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 22:21:15.459211 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.459214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.459255 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.459316 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:15.459759 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.460262 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.461077 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.461289 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.461372 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.462517 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:15.462521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:15.462523 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:15.462544 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:15.464158 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.464576 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.464580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.464582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.464663 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.464929 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.464932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.464934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.465209 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:15.465301 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.465565 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.465676 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:15.484151 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44593
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:15.602181 29699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:15.602492 29699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:15.602540 29699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:15.606724 29699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:15.606796 29699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:15.606837 29699 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:15.606902 29699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:15.612500 29699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:44593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:15.613670 29699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:15.614755 29699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:15.625783 29707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:15.625874 29699 server_base.cc:1034] running on GCE node
W20260811 22:21:15.625800 29709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:15.625789 29706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:15.626514 29699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:15.627275 29699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:15.628439 29699 hybrid_clock.cc:648] HybridClock initialized: now 1786486875628418 us; error 33 us; skew 500 ppm
Aug 11 22:21:15 dist-test-slave-5bn2 krb5kdc[28994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:15.631331 29699 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:15.632308 29699 webserver.cc:460] Webserver started at http://127.26.125.3:33301/ using document root <none> and password file <none>
I20260811 22:21:15.632885 29699 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:15.632956 29699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:15.633201 29699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:15.635488 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "4812d22e302608ae8fc3d70075d44396"
server_key_iv: "1f635cafc10fb380bc2528018a3a2f08"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.636060 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "4812d22e302608ae8fc3d70075d44396"
server_key_iv: "1f635cafc10fb380bc2528018a3a2f08"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.639943 29699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:21:15.642516 29721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:15.643788 29699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:21:15.643936 29699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8"
format_stamp: "Formatted at 2026-08-11 22:21:15 on dist-test-slave-5bn2"
server_key: "4812d22e302608ae8fc3d70075d44396"
server_key_iv: "1f635cafc10fb380bc2528018a3a2f08"
server_key_version: "encryptionkey@0"
I20260811 22:21:15.644049 29699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:15.662766 29699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:15.666074 29699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:15.666270 29699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:15.667939 29699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:15.667997 29699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:15.668062 29699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:15.668088 29699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:21:15 dist-test-slave-5bn2 krb5kdc[28994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:15.678273 29699 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:43403
I20260811 22:21:15.678326 29912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:43403 every 8 connection(s)
I20260811 22:21:15.679482 29699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:15.681393 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.668795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:55189 (local address 127.26.125.62:44593)
0811 22:21:15.668948 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.668953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.669891 (+ 938us) server_negotiation.cc:407] Connection header received
0811 22:21:15.671046 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.671049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.671111 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.671187 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:15.672619 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.673145 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.674291 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.674558 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.674654 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.677577 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:15.677581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:15.677584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:15.677612 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:15.679292 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.679834 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.679838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.679840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.679947 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.680350 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.680353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.680354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.680595 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:15.680678 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.681161 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.681269 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:21:15.681964 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 29699
I20260811 22:21:15.682073 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:15.682307 27124 external_mini_cluster.cc:1428] Setting key 6238f8041a0c2284a5e9fd2a5ffe69bc
I20260811 22:21:15.682294 29743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.669232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44593 (local address 127.26.125.3:55189)
0811 22:21:15.669707 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.669761 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.670779 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.671379 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.671386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.671788 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:15.672472 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.672489 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.673265 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.673269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.674172 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.674182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.674721 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.674725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.675004 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.675628 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:15.675645 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:15.677393 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:15.679403 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.679409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.679422 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.679733 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.680060 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.680063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.680065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.680239 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.680688 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:15.680693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:15.680934 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.681094 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.681351 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":104,"threads_started":1}
I20260811 22:21:15.683758 29914 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44593
I20260811 22:21:15.683902 29914 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:15.684460 29914 heartbeater.cc:511] Master 127.26.125.62:44593 requested a full tablet report, sending...
I20260811 22:21:15.685573 29070 ts_manager.cc:194] Registered new tserver with Master: b4ebdbbd46174bfe957ea4f7ec4e85e8 (127.26.125.3:43403)
I20260811 22:21:15.686251 29070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:55189
I20260811 22:21:15.692211 29923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.685764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44593 (local address 127.26.125.3:37247)
0811 22:21:15.686031 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.686043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.686133 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.686338 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.686341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.686592 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:15.686706 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.686713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.687485 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.687488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.688055 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.688064 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.688475 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.688478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.688571 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.689039 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:15.689049 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:15.689349 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:15.690737 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.690740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.690743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.690994 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.691244 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:15.691248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:15.691250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:15.691330 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:15.691681 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:15.691685 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:15.691777 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.691863 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.692020 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":106,"threads_started":1}
I20260811 22:21:15.692199 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.685812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:37247 (local address 127.26.125.62:44593)
0811 22:21:15.685921 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.685924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.686105 (+ 181us) server_negotiation.cc:407] Connection header received
0811 22:21:15.686204 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.686207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.686249 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.686312 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:15.686806 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.687369 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.688159 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.688381 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.688465 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.689473 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:15.689476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:15.689479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:15.689504 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:15.690636 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.691077 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.691081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.691082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.691166 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.691408 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.691412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.691414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.691595 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:15.691680 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.691932 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.692050 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:21:15.696271 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:21:15 dist-test-slave-5bn2 krb5kdc[28994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:21:15 dist-test-slave-5bn2 krb5kdc[28994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:15.722594 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.714972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.26.125.62:44593)
0811 22:21:15.715126 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.715129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.715140 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:21:15.715180 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.715182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.715222 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.715291 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:15.716014 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.716536 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.717328 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.717540 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.717627 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.719895 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:15.719898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:15.719900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:15.719926 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:15.721067 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.721447 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.721450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.721451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.721532 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:15.721777 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:15.721780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:15.721782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:15.721964 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:15.722092 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.722319 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.722432 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:21:15.725458 29070 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:15.728623 29070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:15.744853 29946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.738475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:41461 (local address 127.0.0.1:39088)
0811 22:21:15.739246 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.739263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.739404 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.740382 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.740388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.740411 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:15.740597 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.740605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.741947 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.741951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.743007 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.743018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.744184 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.744187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.744429 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.744449 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.744549 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.744683 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":135,"spinlock_wait_cycles":1664,"thread_start_us":116,"threads_started":1}
I20260811 22:21:15.744891 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.739379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.26.125.2:41461)
0811 22:21:15.739843 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.739858 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.739884 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:21:15.739972 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.739979 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.740208 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.740398 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:15.740727 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.741777 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.743167 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.744070 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.744197 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.744379 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.744598 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.744710 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":213,"threads_started":1}
I20260811 22:21:15.745411 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.738557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.26.125.1:42159)
0811 22:21:15.739709 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.739717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.739741 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:21:15.739825 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.739830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.740105 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.740260 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:15.740745 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.741662 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.743243 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.743821 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.743924 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.744086 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.745138 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.745252 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1039,"thread_start_us":99,"threads_started":1}
I20260811 22:21:15.745817 29950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.739863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43403 (local address 127.0.0.1:49932)
0811 22:21:15.740129 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.740143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.740223 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.741174 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.741177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.741366 (+ 189us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:15.741475 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.741482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.742943 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.742947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.744070 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.744082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.745380 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.745385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.745482 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.745515 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.745597 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.745675 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":110,"threads_started":1}
I20260811 22:21:15.746384 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.740210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.26.125.3:43403)
0811 22:21:15.740695 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.740709 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.740732 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:21:15.740808 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.740813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.741003 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.741186 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:15.741602 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.742802 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.744245 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.745010 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.745128 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.745288 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.746118 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.746220 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":145,"threads_started":1}
I20260811 22:21:15.746058 29947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.738394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:42159 (local address 127.0.0.1:58854)
0811 22:21:15.739167 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.739240 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.739403 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.740410 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.740413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.740426 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:15.740614 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.740620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.741780 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.741784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.743134 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.743144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.744151 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.744155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.744353 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.744432 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.744558 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.745929 (+ 1371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":124,"spinlock_wait_cycles":14336,"thread_start_us":93,"threads_started":1}
I20260811 22:21:15.748888 29554 tablet_service.cc:1467] Processing CreateTablet for tablet 53306091937748b48f0cf5d1f4d5c374 (DEFAULT_TABLE table=test-table [id=128d6cc29b3a44e289bbf391ddeadc3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:15.749702 29808 tablet_service.cc:1467] Processing CreateTablet for tablet 53306091937748b48f0cf5d1f4d5c374 (DEFAULT_TABLE table=test-table [id=128d6cc29b3a44e289bbf391ddeadc3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:15.750259 29554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53306091937748b48f0cf5d1f4d5c374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:15.750918 29808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53306091937748b48f0cf5d1f4d5c374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:15.752574 29299 tablet_service.cc:1467] Processing CreateTablet for tablet 53306091937748b48f0cf5d1f4d5c374 (DEFAULT_TABLE table=test-table [id=128d6cc29b3a44e289bbf391ddeadc3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:15.753410 29299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53306091937748b48f0cf5d1f4d5c374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:15.758936 29954 tablet_bootstrap.cc:492] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: Bootstrap starting.
I20260811 22:21:15.760573 29956 tablet_bootstrap.cc:492] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: Bootstrap starting.
I20260811 22:21:15.761917 29954 tablet_bootstrap.cc:654] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:15.762892 29954 log.cc:824] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:15.762979 29956 tablet_bootstrap.cc:654] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:15.763273 29959 tablet_bootstrap.cc:492] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Bootstrap starting.
I20260811 22:21:15.763760 29956 log.cc:824] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:15.765303 29954 tablet_bootstrap.cc:492] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: No bootstrap required, opened a new log
I20260811 22:21:15.765564 29954 ts_tablet_manager.cc:1397] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:21:15.765580 29959 tablet_bootstrap.cc:654] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:15.766358 29959 log.cc:824] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:15.766958 29956 tablet_bootstrap.cc:492] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: No bootstrap required, opened a new log
I20260811 22:21:15.767190 29956 ts_tablet_manager.cc:1397] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 22:21:15.768282 29959 tablet_bootstrap.cc:492] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: No bootstrap required, opened a new log
I20260811 22:21:15.768468 29959 ts_tablet_manager.cc:1397] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:21:15.770339 29954 raft_consensus.cc:348] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.770648 29954 raft_consensus.cc:374] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:15.770741 29954 raft_consensus.cc:729] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490733fb2ae24022bc1ae09c0b7a867a, State: Initialized, Role: FOLLOWER
I20260811 22:21:15.771114 29956 raft_consensus.cc:348] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.771253 29954 consensus_queue.cc:260] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.771378 29956 raft_consensus.cc:374] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:15.771456 29956 raft_consensus.cc:729] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a89822a4574a09ac03f3f7bf87b064, State: Initialized, Role: FOLLOWER
I20260811 22:21:15.771557 29959 raft_consensus.cc:348] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.771966 29959 raft_consensus.cc:374] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:15.771908 29956 consensus_queue.cc:260] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.772017 29417 heartbeater.cc:503] Master 127.26.125.62:44593 was elected leader, sending a full tablet report...
I20260811 22:21:15.772087 29959 raft_consensus.cc:729] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4ebdbbd46174bfe957ea4f7ec4e85e8, State: Initialized, Role: FOLLOWER
I20260811 22:21:15.772320 29954 ts_tablet_manager.cc:1428] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:21:15.772543 29959 consensus_queue.cc:260] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.772914 29956 ts_tablet_manager.cc:1428] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260811 22:21:15.773510 29677 heartbeater.cc:503] Master 127.26.125.62:44593 was elected leader, sending a full tablet report...
I20260811 22:21:15.773761 29959 ts_tablet_manager.cc:1428] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:21:15.774135 29914 heartbeater.cc:503] Master 127.26.125.62:44593 was elected leader, sending a full tablet report...
I20260811 22:21:15.829887 29964 raft_consensus.cc:482] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:15.830159 29964 raft_consensus.cc:504] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.831571 29964 leader_election.cc:288] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40a89822a4574a09ac03f3f7bf87b064 (127.26.125.2:41461), b4ebdbbd46174bfe957ea4f7ec4e85e8 (127.26.125.3:43403)
I20260811 22:21:15.837116 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.832025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:59367 (local address 127.26.125.2:41461)
0811 22:21:15.832161 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.832166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.832499 (+ 333us) server_negotiation.cc:407] Connection header received
0811 22:21:15.832614 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.832616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.832672 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.832749 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:15.833133 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.833955 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.835748 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.836392 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.836540 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.836587 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.836894 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.836970 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:21:15.837116 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.832078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:52623 (local address 127.26.125.3:43403)
0811 22:21:15.832186 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.832190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.832436 (+ 246us) server_negotiation.cc:407] Connection header received
0811 22:21:15.832633 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.832636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.832680 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.832751 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:15.833133 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.833935 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.835747 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.836373 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.836512 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.836573 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.836894 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.836970 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:21:15.837182 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.831893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:41461 (local address 127.26.125.1:59367)
0811 22:21:15.832397 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.832408 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.832513 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.832772 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.832774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.832815 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:15.833023 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.833031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.834095 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.834099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.835611 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.835624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.836555 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.836559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.836715 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.836789 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.836876 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.836971 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":213,"spinlock_wait_cycles":50816,"thread_start_us":128,"threads_started":1}
I20260811 22:21:15.837181 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.831875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43403 (local address 127.26.125.1:52623)
0811 22:21:15.832331 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:15.832348 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:15.832491 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:15.832772 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:15.832775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:15.832814 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:15.833023 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.833031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.834095 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.834099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.835611 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:15.835625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.836555 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.836559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:15.836733 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.836793 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 22:21:15.836876 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:21:15.836983 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":194,"spinlock_wait_cycles":26752,"thread_start_us":163,"threads_started":1}
I20260811 22:21:15.837901 29597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53306091937748b48f0cf5d1f4d5c374" candidate_uuid: "490733fb2ae24022bc1ae09c0b7a867a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a89822a4574a09ac03f3f7bf87b064" is_pre_election: true
I20260811 22:21:15.838198 29845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53306091937748b48f0cf5d1f4d5c374" candidate_uuid: "490733fb2ae24022bc1ae09c0b7a867a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" is_pre_election: true
I20260811 22:21:15.838267 29597 raft_consensus.cc:2454] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 490733fb2ae24022bc1ae09c0b7a867a in term 0.
I20260811 22:21:15.838500 29845 raft_consensus.cc:2454] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 490733fb2ae24022bc1ae09c0b7a867a in term 0.
I20260811 22:21:15.838938 29220 leader_election.cc:302] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 40a89822a4574a09ac03f3f7bf87b064, 490733fb2ae24022bc1ae09c0b7a867a, b4ebdbbd46174bfe957ea4f7ec4e85e8; no voters:
I20260811 22:21:15.839262 29964 raft_consensus.cc:2781] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:15.839350 29964 raft_consensus.cc:482] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:15.839402 29964 raft_consensus.cc:3037] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:15.840548 29964 raft_consensus.cc:504] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.840902 29964 leader_election.cc:288] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [CANDIDATE]: Term 1 election: Requested vote from peers 40a89822a4574a09ac03f3f7bf87b064 (127.26.125.2:41461), b4ebdbbd46174bfe957ea4f7ec4e85e8 (127.26.125.3:43403)
I20260811 22:21:15.841323 29597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53306091937748b48f0cf5d1f4d5c374" candidate_uuid: "490733fb2ae24022bc1ae09c0b7a867a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a89822a4574a09ac03f3f7bf87b064"
I20260811 22:21:15.841465 29597 raft_consensus.cc:3037] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:15.841513 29845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53306091937748b48f0cf5d1f4d5c374" candidate_uuid: "490733fb2ae24022bc1ae09c0b7a867a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8"
I20260811 22:21:15.841653 29845 raft_consensus.cc:3037] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:15.843080 29597 raft_consensus.cc:2454] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 490733fb2ae24022bc1ae09c0b7a867a in term 1.
I20260811 22:21:15.843171 29845 raft_consensus.cc:2454] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 490733fb2ae24022bc1ae09c0b7a867a in term 1.
I20260811 22:21:15.843412 29220 leader_election.cc:302] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a89822a4574a09ac03f3f7bf87b064, 490733fb2ae24022bc1ae09c0b7a867a; no voters:
I20260811 22:21:15.843658 29964 raft_consensus.cc:2781] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:15.843915 29964 raft_consensus.cc:686] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 LEADER]: Becoming Leader. State: Replica: 490733fb2ae24022bc1ae09c0b7a867a, State: Running, Role: LEADER
I20260811 22:21:15.844269 29964 consensus_queue.cc:237] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.848078 29068 catalog_manager.cc:5495] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a reported cstate change: term changed from 0 to 1, leader changed from <none> to 490733fb2ae24022bc1ae09c0b7a867a (127.26.125.1). New cstate: current_term: 1 leader_uuid: "490733fb2ae24022bc1ae09c0b7a867a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } health_report { overall_health: UNKNOWN } } }
I20260811 22:21:15.880813 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:15.876558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.26.125.1:42159)
0811 22:21:15.876681 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:15.876684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:15.876751 (+ 67us) server_negotiation.cc:407] Connection header received
0811 22:21:15.876990 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:15.876993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:15.877055 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:15.877133 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:15.877357 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.877893 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.878912 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:15.879282 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:15.879389 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:15.879765 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:15.880390 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:15.880525 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:15.880570 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 22:21:15.880646 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:21:15.888103 29597 raft_consensus.cc:1261] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Refusing update from remote peer 490733fb2ae24022bc1ae09c0b7a867a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:15.888103 29845 raft_consensus.cc:1261] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Refusing update from remote peer 490733fb2ae24022bc1ae09c0b7a867a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:15.888852 29972 consensus_queue.cc:1035] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:15.889129 29964 consensus_queue.cc:1035] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:15.898286 29978 mvcc.cc:205] Tried to move back new op lower bound from 7317450243629903872 to 7317450243461357568. Current Snapshot: MvccSnapshot[applied={T|T < 7317450243629903872}]
I20260811 22:21:15.899495 29977 mvcc.cc:205] Tried to move back new op lower bound from 7317450243629903872 to 7317450243461357568. Current Snapshot: MvccSnapshot[applied={T|T < 7317450243629903872 or (T in {7317450243629903872})}]
I20260811 22:21:15.900110 29980 mvcc.cc:205] Tried to move back new op lower bound from 7317450243629903872 to 7317450243461357568. Current Snapshot: MvccSnapshot[applied={T|T < 7317450243629903872}]
I20260811 22:21:15.911360 29068 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:15.911701 29068 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:15.914996 29068 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9: Sending DeleteTablet for 3 replicas of tablet 53306091937748b48f0cf5d1f4d5c374
I20260811 22:21:15.915990 29299 tablet_service.cc:1514] Processing DeleteTablet for tablet 53306091937748b48f0cf5d1f4d5c374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:15 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:58854
I20260811 22:21:15.915973 29554 tablet_service.cc:1514] Processing DeleteTablet for tablet 53306091937748b48f0cf5d1f4d5c374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:15 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:39088
I20260811 22:21:15.916247 29808 tablet_service.cc:1514] Processing DeleteTablet for tablet 53306091937748b48f0cf5d1f4d5c374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:15 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:49932
I20260811 22:21:15.916569 29999 tablet_replica.cc:330] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: stopping tablet replica
I20260811 22:21:15.916846 30000 tablet_replica.cc:330] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: stopping tablet replica
I20260811 22:21:15.916971 29999 raft_consensus.cc:2229] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:15.917286 30000 raft_consensus.cc:2229] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:15.917343 29999 raft_consensus.cc:2258] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:15.917760 30000 raft_consensus.cc:2258] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:15.917852 29068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:15.918352 29068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:15.919377 29999 ts_tablet_manager.cc:1905] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:15.919766 30000 ts_tablet_manager.cc:1905] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:15.922271 29999 ts_tablet_manager.cc:1918] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:15.922389 29999 log.cc:1196] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/53306091937748b48f0cf5d1f4d5c374
I20260811 22:21:15.922617 30000 ts_tablet_manager.cc:1918] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:15.922734 29999 ts_tablet_manager.cc:1939] T 53306091937748b48f0cf5d1f4d5c374 P 40a89822a4574a09ac03f3f7bf87b064: Deleting consensus metadata
I20260811 22:21:15.922832 30000 log.cc:1196] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/53306091937748b48f0cf5d1f4d5c374
I20260811 22:21:15.923074 29998 tablet_replica.cc:330] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: stopping tablet replica
I20260811 22:21:15.923269 30000 ts_tablet_manager.cc:1939] T 53306091937748b48f0cf5d1f4d5c374 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Deleting consensus metadata
I20260811 22:21:15.923429 29998 raft_consensus.cc:2229] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:15.923938 29998 raft_consensus.cc:2258] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:15.924423 29299 tablet_service.cc:1467] Processing CreateTablet for tablet e87bb25d242f4877aff5b14dcab91324 (DEFAULT_TABLE table=test-table [id=4e9c12337ec3449d8dd30756bb2bd26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:15.924721 29299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87bb25d242f4877aff5b14dcab91324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:15.925640 29998 ts_tablet_manager.cc:1905] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:15.927338 29954 tablet_bootstrap.cc:492] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: Bootstrap starting.
I20260811 22:21:15.927870 29998 ts_tablet_manager.cc:1918] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:15.927856 29554 tablet_service.cc:1467] Processing CreateTablet for tablet e87bb25d242f4877aff5b14dcab91324 (DEFAULT_TABLE table=test-table [id=4e9c12337ec3449d8dd30756bb2bd26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:15.927953 29998 log.cc:1196] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/53306091937748b48f0cf5d1f4d5c374
I20260811 22:21:15.928115 29554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87bb25d242f4877aff5b14dcab91324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:15.928231 29998 ts_tablet_manager.cc:1939] T 53306091937748b48f0cf5d1f4d5c374 P 490733fb2ae24022bc1ae09c0b7a867a: Deleting consensus metadata
I20260811 22:21:15.928296 29954 tablet_bootstrap.cc:654] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:15.929584 29954 tablet_bootstrap.cc:492] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: No bootstrap required, opened a new log
I20260811 22:21:15.929666 29954 ts_tablet_manager.cc:1397] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:15.930059 29954 raft_consensus.cc:348] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.930156 29954 raft_consensus.cc:374] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:15.930176 29954 raft_consensus.cc:729] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490733fb2ae24022bc1ae09c0b7a867a, State: Initialized, Role: FOLLOWER
I20260811 22:21:15.930299 29954 consensus_queue.cc:260] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.930501 29956 tablet_bootstrap.cc:492] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: Bootstrap starting.
I20260811 22:21:15.930547 29954 ts_tablet_manager.cc:1428] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:15.931595 29956 tablet_bootstrap.cc:654] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:15.932365 29050 catalog_manager.cc:4841] TS 40a89822a4574a09ac03f3f7bf87b064 (127.26.125.2:41461): tablet 53306091937748b48f0cf5d1f4d5c374 (table test-table [id=128d6cc29b3a44e289bbf391ddeadc3d]) successfully deleted
I20260811 22:21:15.933002 29044 catalog_manager.cc:4841] TS 490733fb2ae24022bc1ae09c0b7a867a (127.26.125.1:42159): tablet 53306091937748b48f0cf5d1f4d5c374 (table test-table [id=128d6cc29b3a44e289bbf391ddeadc3d]) successfully deleted
I20260811 22:21:15.933030 29042 catalog_manager.cc:4841] TS b4ebdbbd46174bfe957ea4f7ec4e85e8 (127.26.125.3:43403): tablet 53306091937748b48f0cf5d1f4d5c374 (table test-table [id=128d6cc29b3a44e289bbf391ddeadc3d]) successfully deleted
I20260811 22:21:15.933455 29808 tablet_service.cc:1467] Processing CreateTablet for tablet e87bb25d242f4877aff5b14dcab91324 (DEFAULT_TABLE table=test-table [id=4e9c12337ec3449d8dd30756bb2bd26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:15.933758 29808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87bb25d242f4877aff5b14dcab91324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:15.936430 29959 tablet_bootstrap.cc:492] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Bootstrap starting.
I20260811 22:21:15.937201 29956 tablet_bootstrap.cc:492] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: No bootstrap required, opened a new log
I20260811 22:21:15.937309 29956 ts_tablet_manager.cc:1397] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 22:21:15.937476 29959 tablet_bootstrap.cc:654] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:15.937867 29956 raft_consensus.cc:348] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.937994 29956 raft_consensus.cc:374] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:15.938025 29956 raft_consensus.cc:729] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a89822a4574a09ac03f3f7bf87b064, State: Initialized, Role: FOLLOWER
I20260811 22:21:15.938163 29956 consensus_queue.cc:260] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.938457 29956 ts_tablet_manager.cc:1428] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:15.940291 29959 tablet_bootstrap.cc:492] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: No bootstrap required, opened a new log
I20260811 22:21:15.940372 29959 ts_tablet_manager.cc:1397] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:21:15.940821 29959 raft_consensus.cc:348] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.940919 29959 raft_consensus.cc:374] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:15.940943 29959 raft_consensus.cc:729] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4ebdbbd46174bfe957ea4f7ec4e85e8, State: Initialized, Role: FOLLOWER
I20260811 22:21:15.941059 29959 consensus_queue.cc:260] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:15.941280 29959 ts_tablet_manager.cc:1428] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260811 22:21:15.951387 29680 tablet.cc:2369] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:21:15.978240 29420 tablet.cc:2369] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:16.059746 29969 raft_consensus.cc:482] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:16.059963 29969 raft_consensus.cc:504] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.061601 29969 leader_election.cc:288] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 490733fb2ae24022bc1ae09c0b7a867a (127.26.125.1:42159), 40a89822a4574a09ac03f3f7bf87b064 (127.26.125.2:41461)
I20260811 22:21:16.066481 29743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.061859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:41461 (local address 127.26.125.3:58891)
0811 22:21:16.062034 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:16.062056 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:16.062184 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:16.062813 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:16.062816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:16.062831 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:16.062977 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.062985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.064134 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.064138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.065284 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.065296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.065995 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.065999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.066119 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.066143 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:21:16.066212 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:21:16.066294 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":20}
I20260811 22:21:16.067039 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.062207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:42335 (local address 127.26.125.1:42159)
0811 22:21:16.062341 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.062362 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.062581 (+ 219us) server_negotiation.cc:407] Connection header received
0811 22:21:16.062721 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.062724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.062781 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.062892 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:16.063263 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.063769 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.065446 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.066118 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.066253 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.066317 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.066753 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.066887 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:21:16.067062 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.062423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:58891 (local address 127.26.125.2:41461)
0811 22:21:16.062595 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.062600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.062614 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:16.062672 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.062675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.062722 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.062788 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:16.063109 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.063985 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.065456 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.065871 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.065995 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.066056 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.066813 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.066940 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:21:16.067497 30007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.061859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:42159 (local address 127.26.125.3:42335)
0811 22:21:16.062473 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:16.062487 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:16.062604 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:16.062941 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:16.062943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:16.062970 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:16.063101 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.063108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.063888 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.063892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.065284 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.065296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.066431 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.066437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.066573 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.066607 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:21:16.067218 (+ 611us) client_negotiation.cc:240] Negotiation successful
0811 22:21:16.067311 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":198,"thread_start_us":79,"threads_started":1}
I20260811 22:21:16.067484 29597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87bb25d242f4877aff5b14dcab91324" candidate_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a89822a4574a09ac03f3f7bf87b064" is_pre_election: true
I20260811 22:21:16.067646 29597 raft_consensus.cc:2454] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4ebdbbd46174bfe957ea4f7ec4e85e8 in term 0.
I20260811 22:21:16.068233 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87bb25d242f4877aff5b14dcab91324" candidate_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490733fb2ae24022bc1ae09c0b7a867a" is_pre_election: true
I20260811 22:21:16.068230 29727 leader_election.cc:302] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a89822a4574a09ac03f3f7bf87b064, b4ebdbbd46174bfe957ea4f7ec4e85e8; no voters:
I20260811 22:21:16.068462 29331 raft_consensus.cc:2454] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4ebdbbd46174bfe957ea4f7ec4e85e8 in term 0.
I20260811 22:21:16.068557 29969 raft_consensus.cc:2781] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:16.068617 29969 raft_consensus.cc:482] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:16.068656 29969 raft_consensus.cc:3037] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:16.069774 29969 raft_consensus.cc:504] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.070161 29969 leader_election.cc:288] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [CANDIDATE]: Term 1 election: Requested vote from peers 490733fb2ae24022bc1ae09c0b7a867a (127.26.125.1:42159), 40a89822a4574a09ac03f3f7bf87b064 (127.26.125.2:41461)
I20260811 22:21:16.070513 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87bb25d242f4877aff5b14dcab91324" candidate_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490733fb2ae24022bc1ae09c0b7a867a"
I20260811 22:21:16.070572 29597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87bb25d242f4877aff5b14dcab91324" candidate_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a89822a4574a09ac03f3f7bf87b064"
I20260811 22:21:16.070634 29331 raft_consensus.cc:3037] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:16.070724 29597 raft_consensus.cc:3037] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:16.071662 29331 raft_consensus.cc:2454] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4ebdbbd46174bfe957ea4f7ec4e85e8 in term 1.
I20260811 22:21:16.071952 29597 raft_consensus.cc:2454] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4ebdbbd46174bfe957ea4f7ec4e85e8 in term 1.
I20260811 22:21:16.072010 29724 leader_election.cc:302] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490733fb2ae24022bc1ae09c0b7a867a, b4ebdbbd46174bfe957ea4f7ec4e85e8; no voters:
I20260811 22:21:16.072240 29969 raft_consensus.cc:2781] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:16.072520 29969 raft_consensus.cc:686] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 LEADER]: Becoming Leader. State: Replica: b4ebdbbd46174bfe957ea4f7ec4e85e8, State: Running, Role: LEADER
I20260811 22:21:16.072808 29969 consensus_queue.cc:237] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.075441 29068 catalog_manager.cc:5495] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4ebdbbd46174bfe957ea4f7ec4e85e8 (127.26.125.3). New cstate: current_term: 1 leader_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } health_report { overall_health: HEALTHY } } }
Aug 11 22:21:16 dist-test-slave-5bn2 krb5kdc[28994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486874, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:16.145299 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.136824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44593 (local address 127.0.0.1:52940)
0811 22:21:16.137170 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:16.137183 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:16.137304 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:16.137561 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:16.137564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:16.138027 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:16.138125 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.138132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.138884 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.138887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.139445 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.139454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.139868 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.139872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.139984 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.140574 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:16.140587 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:16.142124 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:16.143729 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:16.143741 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:16.143750 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:16.143991 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:16.144238 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:16.144242 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:16.144244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:16.144319 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:16.144687 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:16.144696 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:16.144816 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 22:21:16.144915 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 22:21:16.145085 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":5,"thread_start_us":121,"threads_started":1}
I20260811 22:21:16.145299 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.136913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52940 (local address 127.26.125.62:44593)
0811 22:21:16.137096 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.137101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.137259 (+ 158us) server_negotiation.cc:407] Connection header received
0811 22:21:16.137386 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.137389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.137426 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.137490 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:16.138241 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.138757 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.139562 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.139768 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.139855 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.142256 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:16.142259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:16.142261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:16.142282 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:16.143629 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:16.144081 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:16.144085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:16.144086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:16.144160 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:16.144394 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:16.144398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:16.144399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:16.144587 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:16.144673 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.144975 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.145152 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":81}
I20260811 22:21:16.150098 29068 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:52940:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:21:16.150923 29068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:16.156268 29299 tablet_service.cc:1467] Processing CreateTablet for tablet 4951c55531e2481cb928b98e79ae0aa2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec75e41b160f46d18f8b721f8790ac83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:16.156268 29554 tablet_service.cc:1467] Processing CreateTablet for tablet 4951c55531e2481cb928b98e79ae0aa2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec75e41b160f46d18f8b721f8790ac83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:16.156320 29808 tablet_service.cc:1467] Processing CreateTablet for tablet 4951c55531e2481cb928b98e79ae0aa2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec75e41b160f46d18f8b721f8790ac83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:16.156725 29554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4951c55531e2481cb928b98e79ae0aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:16.156725 29299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4951c55531e2481cb928b98e79ae0aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:16.156729 29808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4951c55531e2481cb928b98e79ae0aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:16.159713 29956 tablet_bootstrap.cc:492] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: Bootstrap starting.
I20260811 22:21:16.159713 29959 tablet_bootstrap.cc:492] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Bootstrap starting.
I20260811 22:21:16.159713 29954 tablet_bootstrap.cc:492] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: Bootstrap starting.
I20260811 22:21:16.160691 29956 tablet_bootstrap.cc:654] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:16.160701 29954 tablet_bootstrap.cc:654] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:16.160701 29959 tablet_bootstrap.cc:654] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:16.161968 29956 tablet_bootstrap.cc:492] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: No bootstrap required, opened a new log
I20260811 22:21:16.162066 29956 ts_tablet_manager.cc:1397] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 22:21:16.162417 29954 tablet_bootstrap.cc:492] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: No bootstrap required, opened a new log
I20260811 22:21:16.162511 29954 ts_tablet_manager.cc:1397] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:16.162565 29956 raft_consensus.cc:348] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.162688 29956 raft_consensus.cc:374] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:16.162712 29956 raft_consensus.cc:729] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a89822a4574a09ac03f3f7bf87b064, State: Initialized, Role: FOLLOWER
I20260811 22:21:16.162837 29956 consensus_queue.cc:260] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.162959 29954 raft_consensus.cc:348] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.163076 29954 raft_consensus.cc:374] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:16.163115 29954 raft_consensus.cc:729] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490733fb2ae24022bc1ae09c0b7a867a, State: Initialized, Role: FOLLOWER
I20260811 22:21:16.163132 29956 ts_tablet_manager.cc:1428] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:21:16.163237 29954 consensus_queue.cc:260] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.163185 29968 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.163347 29968 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:16.163483 29954 ts_tablet_manager.cc:1428] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:16.164059 29972 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.164201 29972 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:16.164264 29959 tablet_bootstrap.cc:492] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: No bootstrap required, opened a new log
I20260811 22:21:16.164337 29959 ts_tablet_manager.cc:1397] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:21:16.164799 29959 raft_consensus.cc:348] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.164925 29959 raft_consensus.cc:374] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:16.164965 29959 raft_consensus.cc:729] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4ebdbbd46174bfe957ea4f7ec4e85e8, State: Initialized, Role: FOLLOWER
I20260811 22:21:16.165134 29959 consensus_queue.cc:260] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.165374 29969 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.165513 29969 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:16.165772 29959 ts_tablet_manager.cc:1428] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260811 22:21:16.181013 29915 tablet.cc:2369] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:16.190219 29968 raft_consensus.cc:482] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:16.190367 29968 raft_consensus.cc:504] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.191517 29968 leader_election.cc:288] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 490733fb2ae24022bc1ae09c0b7a867a (127.26.125.1:42159), b4ebdbbd46174bfe957ea4f7ec4e85e8 (127.26.125.3:43403)
I20260811 22:21:16.196064 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.191862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:56529 (local address 127.26.125.3:43403)
0811 22:21:16.191994 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.191999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.192369 (+ 370us) server_negotiation.cc:407] Connection header received
0811 22:21:16.192539 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.192543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.192594 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.192641 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:16.193005 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.193750 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.194945 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.195447 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.195557 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.195605 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.195861 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.195926 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:21:16.196169 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.191875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:54333 (local address 127.26.125.1:42159)
0811 22:21:16.192048 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.192052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.192179 (+ 127us) server_negotiation.cc:407] Connection header received
0811 22:21:16.192401 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.192405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.192461 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.192508 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:16.192933 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.193753 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.194988 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.195492 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.195603 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.195652 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.195948 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.196034 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:21:16.196396 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.191741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43403 (local address 127.26.125.2:56529)
0811 22:21:16.192270 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:16.192282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:16.192391 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:16.192782 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:16.192785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:16.192803 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:16.192898 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.192902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.193872 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.193874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.194793 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.194804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.195661 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.195664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.195752 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.195773 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:21:16.196194 (+ 421us) client_negotiation.cc:240] Negotiation successful
0811 22:21:16.196264 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":227,"thread_start_us":82,"threads_started":1}
I20260811 22:21:16.196568 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.191741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:42159 (local address 127.26.125.2:54333)
0811 22:21:16.192096 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:16.192111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:16.192265 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:16.192665 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:16.192668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:16.192687 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:16.192817 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.192824 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.193902 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.193905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.194802 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.194812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.195714 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.195717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.195814 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.195837 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:21:16.196359 (+ 522us) client_negotiation.cc:240] Negotiation successful
0811 22:21:16.196434 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":60,"thread_start_us":74,"threads_started":1}
I20260811 22:21:16.196982 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4951c55531e2481cb928b98e79ae0aa2" candidate_uuid: "40a89822a4574a09ac03f3f7bf87b064" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490733fb2ae24022bc1ae09c0b7a867a" is_pre_election: true
I20260811 22:21:16.197165 29331 raft_consensus.cc:2454] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a89822a4574a09ac03f3f7bf87b064 in term 0.
I20260811 22:21:16.197497 29845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4951c55531e2481cb928b98e79ae0aa2" candidate_uuid: "40a89822a4574a09ac03f3f7bf87b064" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" is_pre_election: true
I20260811 22:21:16.197642 29845 raft_consensus.cc:2454] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a89822a4574a09ac03f3f7bf87b064 in term 0.
I20260811 22:21:16.197613 29457 leader_election.cc:302] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a89822a4574a09ac03f3f7bf87b064, 490733fb2ae24022bc1ae09c0b7a867a; no voters:
I20260811 22:21:16.197914 29968 raft_consensus.cc:2781] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:16.197978 29968 raft_consensus.cc:482] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:16.198014 29968 raft_consensus.cc:3037] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:16.199116 29968 raft_consensus.cc:504] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.199471 29968 leader_election.cc:288] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [CANDIDATE]: Term 1 election: Requested vote from peers 490733fb2ae24022bc1ae09c0b7a867a (127.26.125.1:42159), b4ebdbbd46174bfe957ea4f7ec4e85e8 (127.26.125.3:43403)
I20260811 22:21:16.199852 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4951c55531e2481cb928b98e79ae0aa2" candidate_uuid: "40a89822a4574a09ac03f3f7bf87b064" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490733fb2ae24022bc1ae09c0b7a867a"
I20260811 22:21:16.199848 29845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4951c55531e2481cb928b98e79ae0aa2" candidate_uuid: "40a89822a4574a09ac03f3f7bf87b064" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8"
I20260811 22:21:16.199971 29845 raft_consensus.cc:3037] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:16.199978 29331 raft_consensus.cc:3037] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:16.201148 29845 raft_consensus.cc:2454] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a89822a4574a09ac03f3f7bf87b064 in term 1.
I20260811 22:21:16.201205 29331 raft_consensus.cc:2454] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a89822a4574a09ac03f3f7bf87b064 in term 1.
I20260811 22:21:16.201521 29457 leader_election.cc:302] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a89822a4574a09ac03f3f7bf87b064, 490733fb2ae24022bc1ae09c0b7a867a; no voters:
I20260811 22:21:16.201692 29968 raft_consensus.cc:2781] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:16.201876 29968 raft_consensus.cc:686] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 LEADER]: Becoming Leader. State: Replica: 40a89822a4574a09ac03f3f7bf87b064, State: Running, Role: LEADER
I20260811 22:21:16.202168 29968 consensus_queue.cc:237] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } }
I20260811 22:21:16.203543 30028 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: TxnStatusTablet state changed. Reason: New leader 40a89822a4574a09ac03f3f7bf87b064. Latest consensus state: current_term: 1 leader_uuid: "40a89822a4574a09ac03f3f7bf87b064" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.203661 30028 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:16.204000 30030 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:21:16.204221 30030 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:21:16.204612 29067 catalog_manager.cc:5495] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40a89822a4574a09ac03f3f7bf87b064 (127.26.125.2). New cstate: current_term: 1 leader_uuid: "40a89822a4574a09ac03f3f7bf87b064" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } health_report { overall_health: UNKNOWN } } }
I20260811 22:21:16.220877 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.217290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39096 (local address 127.26.125.2:41461)
0811 22:21:16.217424 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.217429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.217444 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:16.217530 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.217533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.217590 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.217668 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:16.217897 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.218410 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.219347 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.219607 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.219696 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.219987 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:16.220488 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:16.220593 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.220674 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.220728 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:16.220877 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.217130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:41461 (local address 127.0.0.1:39096)
0811 22:21:16.217280 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:16.217295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:16.217405 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:16.217692 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:16.217695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:16.217702 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:16.217802 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.217807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.218513 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.218516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.219242 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.219253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.219710 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.219714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.219800 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.219849 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:16.220581 (+ 732us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:16.220599 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:21:16.220665 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:21:16.220756 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 22:21:16.224184 29845 raft_consensus.cc:1261] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Refusing update from remote peer 40a89822a4574a09ac03f3f7bf87b064: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:16.224288 29331 raft_consensus.cc:1261] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Refusing update from remote peer 40a89822a4574a09ac03f3f7bf87b064: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:16.224726 29968 consensus_queue.cc:1035] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:16.224923 30028 consensus_queue.cc:1035] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064 [LEADER]: Connected to new peer: Peer: permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:16.226322 29969 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: TxnStatusTablet state changed. Reason: New leader 40a89822a4574a09ac03f3f7bf87b064. Latest consensus state: current_term: 1 leader_uuid: "40a89822a4574a09ac03f3f7bf87b064" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.226439 29969 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:16.227169 29972 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: TxnStatusTablet state changed. Reason: New leader 40a89822a4574a09ac03f3f7bf87b064. Latest consensus state: current_term: 1 leader_uuid: "40a89822a4574a09ac03f3f7bf87b064" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.227274 29972 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:16.228173 29968 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "40a89822a4574a09ac03f3f7bf87b064" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.228283 29968 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:16.230170 29969 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "40a89822a4574a09ac03f3f7bf87b064" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.230278 29969 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P b4ebdbbd46174bfe957ea4f7ec4e85e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:16.235875 29968 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "40a89822a4574a09ac03f3f7bf87b064" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.236008 29968 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P 40a89822a4574a09ac03f3f7bf87b064: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:16.238204 29972 tablet_replica.cc:439] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "40a89822a4574a09ac03f3f7bf87b064" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 } } peers { permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 } } peers { permanent_uuid: "b4ebdbbd46174bfe957ea4f7ec4e85e8" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43403 } } }
I20260811 22:21:16.238327 29972 tablet_replica.cc:442] T 4951c55531e2481cb928b98e79ae0aa2 P 490733fb2ae24022bc1ae09c0b7a867a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:16.241416 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.236681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.26.125.3:43403)
0811 22:21:16.236818 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.236823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.236838 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:16.236918 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.236921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.236994 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.237112 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:16.237489 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.238277 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.239526 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.239864 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.239955 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.240339 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:16.240931 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:16.241040 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.241199 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.241270 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:16.253679 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.249888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:35819 (local address 127.26.125.2:41461)
0811 22:21:16.250104 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.250108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.250203 (+ 95us) server_negotiation.cc:407] Connection header received
0811 22:21:16.250347 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.250350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.250400 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.250466 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:16.250771 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.251353 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.252639 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.252860 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.252941 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.253273 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:16.253354 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:16.253402 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.253504 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.253557 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260811 22:21:16.253772 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.249744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:41461 (local address 127.26.125.3:35819)
0811 22:21:16.250110 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:16.250126 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:16.250258 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:16.250489 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:16.250492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:16.250501 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:16.250635 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.250641 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.251457 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.251461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.252502 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.252524 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.252968 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.252971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.253073 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.253147 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:16.253422 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:16.253429 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:21:16.253488 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 22:21:16.253594 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":127,"threads_started":1}
I20260811 22:21:16.263373 29597 raft_consensus.cc:1261] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Refusing update from remote peer b4ebdbbd46174bfe957ea4f7ec4e85e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:16.263373 29331 raft_consensus.cc:1261] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Refusing update from remote peer b4ebdbbd46174bfe957ea4f7ec4e85e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:16.263912 29969 consensus_queue.cc:1035] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a89822a4574a09ac03f3f7bf87b064" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:16.264173 30008 consensus_queue.cc:1035] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "490733fb2ae24022bc1ae09c0b7a867a" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:16.290046 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.285939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:51421 (local address 127.26.125.3:43403)
0811 22:21:16.286068 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.286073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.286296 (+ 223us) server_negotiation.cc:407] Connection header received
0811 22:21:16.286452 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.286455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.286517 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.286596 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:16.286998 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.287734 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.289004 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.289232 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.289308 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.289633 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:16.289710 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:16.289761 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.289867 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.289917 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:21:16.290297 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.285828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43403 (local address 127.26.125.2:51421)
0811 22:21:16.286210 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:16.286225 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:16.286348 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:16.286711 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:16.286714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:16.286722 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:16.286873 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.286880 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.287878 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.287881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.288870 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:16.288881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.289329 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.289332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:16.289432 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.289495 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:16.289780 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:16.289786 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:21:16.289845 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 22:21:16.289932 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":155,"threads_started":1}
I20260811 22:21:16.318954 29067 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:16.319159 29067 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:16.321717 29067 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07c87f321cf34deeabcdf69ce325e7a9: Sending DeleteTablet for 3 replicas of tablet e87bb25d242f4877aff5b14dcab91324
I20260811 22:21:16.322449 29299 tablet_service.cc:1514] Processing DeleteTablet for tablet e87bb25d242f4877aff5b14dcab91324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:16 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:58854
I20260811 22:21:16.322675 29998 tablet_replica.cc:330] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: stopping tablet replica
I20260811 22:21:16.322803 29998 raft_consensus.cc:2229] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:16.322958 29998 raft_consensus.cc:2258] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:16.323436 29808 tablet_service.cc:1514] Processing DeleteTablet for tablet e87bb25d242f4877aff5b14dcab91324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:16 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:49932
I20260811 22:21:16.323632 30000 tablet_replica.cc:330] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: stopping tablet replica
I20260811 22:21:16.323762 30000 raft_consensus.cc:2229] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:16.323865 29998 ts_tablet_manager.cc:1905] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:16.324095 30000 raft_consensus.cc:2258] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:16.327135 29998 ts_tablet_manager.cc:1918] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:16.327224 29998 log.cc:1196] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/e87bb25d242f4877aff5b14dcab91324
I20260811 22:21:16.327479 29998 ts_tablet_manager.cc:1939] T e87bb25d242f4877aff5b14dcab91324 P 490733fb2ae24022bc1ae09c0b7a867a: Deleting consensus metadata
I20260811 22:21:16.327605 30000 ts_tablet_manager.cc:1905] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:16.328269 29044 catalog_manager.cc:4841] TS 490733fb2ae24022bc1ae09c0b7a867a (127.26.125.1:42159): tablet e87bb25d242f4877aff5b14dcab91324 (table test-table [id=4e9c12337ec3449d8dd30756bb2bd26d]) successfully deleted
I20260811 22:21:16.330696 30000 ts_tablet_manager.cc:1918] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:16.330780 30000 log.cc:1196] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/e87bb25d242f4877aff5b14dcab91324
I20260811 22:21:16.332386 29554 tablet_service.cc:1514] Processing DeleteTablet for tablet e87bb25d242f4877aff5b14dcab91324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:16 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:39088
I20260811 22:21:16.332671 29999 tablet_replica.cc:330] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: stopping tablet replica
I20260811 22:21:16.332795 29999 raft_consensus.cc:2229] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:16.332926 29999 raft_consensus.cc:2258] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:16.333487 30000 ts_tablet_manager.cc:1939] T e87bb25d242f4877aff5b14dcab91324 P b4ebdbbd46174bfe957ea4f7ec4e85e8: Deleting consensus metadata
I20260811 22:21:16.334337 29042 catalog_manager.cc:4841] TS b4ebdbbd46174bfe957ea4f7ec4e85e8 (127.26.125.3:43403): tablet e87bb25d242f4877aff5b14dcab91324 (table test-table [id=4e9c12337ec3449d8dd30756bb2bd26d]) successfully deleted
I20260811 22:21:16.335290 29999 ts_tablet_manager.cc:1905] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:16.338318 29999 ts_tablet_manager.cc:1918] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:16.338397 29999 log.cc:1196] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/e87bb25d242f4877aff5b14dcab91324
I20260811 22:21:16.338639 29999 ts_tablet_manager.cc:1939] T e87bb25d242f4877aff5b14dcab91324 P 40a89822a4574a09ac03f3f7bf87b064: Deleting consensus metadata
I20260811 22:21:16.339607 29050 catalog_manager.cc:4841] TS 40a89822a4574a09ac03f3f7bf87b064 (127.26.125.2:41461): tablet e87bb25d242f4877aff5b14dcab91324 (table test-table [id=4e9c12337ec3449d8dd30756bb2bd26d]) successfully deleted
Aug 11 22:21:16 dist-test-slave-5bn2 krb5kdc[28994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486875, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:16.346428 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.332679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.26.125.1:42159)
0811 22:21:16.332801 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.332806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.333469 (+ 663us) server_negotiation.cc:407] Connection header received
0811 22:21:16.333682 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.333686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.333745 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.333801 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:16.334795 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.335595 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.336858 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.337187 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.337265 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.341519 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:16.341537 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:16.341556 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:16.341596 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:16.343856 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:16.344392 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:16.344400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:16.344402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:16.344528 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:16.344884 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:16.344889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:16.344890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:16.345266 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:16.345460 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.345918 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.346231 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 22:21:16.347007 29408 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58880
I20260811 22:21:16.358842 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:16.349022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.26.125.62:44593)
0811 22:21:16.349178 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:16.349183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:16.349698 (+ 515us) server_negotiation.cc:407] Connection header received
0811 22:21:16.349821 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:16.349825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:16.349890 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:16.349980 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:16.350812 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.351679 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.352853 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:16.353194 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:16.353309 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:16.355128 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:16.355132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:16.355135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:16.355166 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:16.356946 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:16.357436 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:16.357440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:16.357442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:16.357574 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:16.357857 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:16.357860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:16.357862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:16.358102 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:16.358233 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:16.358516 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:21:16.358658 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:21:16.359684 29067 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52950
I20260811 22:21:16.361392 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 29191
I20260811 22:21:16.380784 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 29431
I20260811 22:21:16.396142 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 29699
I20260811 22:21:16.410660 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 29013
2026-08-11T22:21:16Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2993 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:16 dist-test-slave-5bn2 krb5kdc[30075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:16 dist-test-slave-5bn2 krb5kdc[30075](info): set up 2 sockets
Aug 11 22:21:16 dist-test-slave-5bn2 krb5kdc[30075](info): commencing operation
krb5kdc: starting...
W20260811 22:21:17.613843 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:17 dist-test-slave-5bn2 krb5kdc[30075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:17.741195 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41713
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:34731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:41713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:17.856858 30099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:17.857158 30099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:17.857198 30099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:17.861259 30099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:17.861333 30099 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:21:17.861348 30099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:17.861366 30099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:17.861380 30099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:17.861394 30099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:17.867228 30099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:34731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:41713
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:17.868340 30099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:17.869407 30099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:17.878225 30105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:17.878235 30112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:17.878225 30108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:17.878280 30099 server_base.cc:1034] running on GCE node
I20260811 22:21:17.879118 30099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:17.880260 30099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:17.881399 30099 hybrid_clock.cc:648] HybridClock initialized: now 1786486877881372 us; error 44 us; skew 500 ppm
Aug 11 22:21:17 dist-test-slave-5bn2 krb5kdc[30075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486877, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:17.885160 30099 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:17.886345 30099 webserver.cc:460] Webserver started at http://127.26.125.62:34619/ using document root <none> and password file <none>
I20260811 22:21:17.887137 30099 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:17.887239 30099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:17.887504 30099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:17.889870 30099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "c893d7be21414c148f4e70188a652c04"
format_stamp: "Formatted at 2026-08-11 22:21:17 on dist-test-slave-5bn2"
server_key: "237f9afee7d0fca65e593a232bb643b2"
server_key_iv: "d5c726a7c1f1efab51d2ad7520650b8e"
server_key_version: "encryptionkey@0"
I20260811 22:21:17.890528 30099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "c893d7be21414c148f4e70188a652c04"
format_stamp: "Formatted at 2026-08-11 22:21:17 on dist-test-slave-5bn2"
server_key: "237f9afee7d0fca65e593a232bb643b2"
server_key_iv: "d5c726a7c1f1efab51d2ad7520650b8e"
server_key_version: "encryptionkey@0"
I20260811 22:21:17.894486 30099 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:21:17.897063 30124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:17.898378 30099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:17.898501 30099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "c893d7be21414c148f4e70188a652c04"
format_stamp: "Formatted at 2026-08-11 22:21:17 on dist-test-slave-5bn2"
server_key: "237f9afee7d0fca65e593a232bb643b2"
server_key_iv: "d5c726a7c1f1efab51d2ad7520650b8e"
server_key_version: "encryptionkey@0"
I20260811 22:21:17.898608 30099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:17.928824 30099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:17.932332 30099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:17.932534 30099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:17.940586 30099 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:41713
I20260811 22:21:17.940608 30231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:41713 every 8 connection(s)
I20260811 22:21:17.941617 30099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:17.944595 30234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:17.948573 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30099
I20260811 22:21:17.948689 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:17.948922 27124 external_mini_cluster.cc:1428] Setting key 0955b0d4cdfad68c74731009019c6998
I20260811 22:21:17.950610 30234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: Bootstrap starting.
I20260811 22:21:17.953195 30234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:17.953919 30234 log.cc:824] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:17.956055 30234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: No bootstrap required, opened a new log
Aug 11 22:21:17 dist-test-slave-5bn2 krb5kdc[30075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:17.959726 30234 raft_consensus.cc:348] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893d7be21414c148f4e70188a652c04" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41713 } }
I20260811 22:21:17.959968 30234 raft_consensus.cc:374] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:17.960027 30234 raft_consensus.cc:729] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c893d7be21414c148f4e70188a652c04, State: Initialized, Role: FOLLOWER
I20260811 22:21:17.960557 30234 consensus_queue.cc:260] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893d7be21414c148f4e70188a652c04" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41713 } }
I20260811 22:21:17.960721 30234 raft_consensus.cc:388] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:17.960796 30234 raft_consensus.cc:482] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:17.960907 30234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:17.961994 30234 raft_consensus.cc:504] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893d7be21414c148f4e70188a652c04" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41713 } }
I20260811 22:21:17.962339 30234 leader_election.cc:302] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c893d7be21414c148f4e70188a652c04; no voters:
I20260811 22:21:17.962661 30234 leader_election.cc:288] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:17.962775 30244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:17.962973 30244 raft_consensus.cc:686] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [term 1 LEADER]: Becoming Leader. State: Replica: c893d7be21414c148f4e70188a652c04, State: Running, Role: LEADER
I20260811 22:21:17.963037 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:17.950348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.26.125.62:41713)
0811 22:21:17.950766 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:17.950781 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:17.950819 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:21:17.951486 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:17.951504 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:17.951886 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:17.952188 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:17.953017 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:17.954066 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:17.955099 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:17.955564 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:17.955677 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:17.958309 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:17.958315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:17.958330 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:17.958358 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:17.959981 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:17.960460 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:17.960465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:17.960469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:17.960583 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:17.960863 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:17.960867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:17.960869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:17.961327 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:17.961491 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:17.961765 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 22:21:17.962062 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":110,"threads_started":1}
I20260811 22:21:17.963302 30244 consensus_queue.cc:237] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893d7be21414c148f4e70188a652c04" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41713 } }
I20260811 22:21:17.964054 30234 sys_catalog.cc:564] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:17.964555 30244 sys_catalog.cc:455] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c893d7be21414c148f4e70188a652c04. Latest consensus state: current_term: 1 leader_uuid: "c893d7be21414c148f4e70188a652c04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893d7be21414c148f4e70188a652c04" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41713 } } }
I20260811 22:21:17.964661 30244 sys_catalog.cc:458] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:17.964928 30247 sys_catalog.cc:455] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c893d7be21414c148f4e70188a652c04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893d7be21414c148f4e70188a652c04" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41713 } } }
I20260811 22:21:17.965034 30247 sys_catalog.cc:458] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:17.965397 30255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:17.968887 30255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:17.975169 30255 catalog_manager.cc:1347] Generated new cluster ID: a60e22f4e72f4d1992d851d51c92b706
I20260811 22:21:17.975265 30255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:17.998237 30255 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:17.999666 30255 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:18.014815 30255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: Generated new TSK 0
I20260811 22:21:18.015890 30255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:18.054454 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41713
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:34731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:18.174966 30279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:18.175261 30279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:18.175310 30279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:18.179299 30279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:18.179356 30279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:18.179397 30279 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:21:18.179418 30279 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:18.179454 30279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:18.184712 30279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:34731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:41713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:18.185772 30279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:18.186807 30279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:18.197027 30286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:18.197029 30292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:18.197029 30288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:18.197392 30279 server_base.cc:1034] running on GCE node
I20260811 22:21:18.197824 30279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:18.198498 30279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:18.199661 30279 hybrid_clock.cc:648] HybridClock initialized: now 1786486878199636 us; error 38 us; skew 500 ppm
Aug 11 22:21:18 dist-test-slave-5bn2 krb5kdc[30075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486878, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:18.202764 30279 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:18.203794 30279 webserver.cc:460] Webserver started at http://127.26.125.1:33933/ using document root <none> and password file <none>
I20260811 22:21:18.204386 30279 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:18.204456 30279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:18.204672 30279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:18.206787 30279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "9236254bb50f4d599bfb9c43830f21c3"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "6cf952bac9453d7c929193190bd6c083"
server_key_iv: "eaa6247a4f851c8967ab183de8cc875c"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.207382 30279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "9236254bb50f4d599bfb9c43830f21c3"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "6cf952bac9453d7c929193190bd6c083"
server_key_iv: "eaa6247a4f851c8967ab183de8cc875c"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.211324 30279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:21:18.213838 30301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:18.215159 30279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:21:18.215272 30279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "9236254bb50f4d599bfb9c43830f21c3"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "6cf952bac9453d7c929193190bd6c083"
server_key_iv: "eaa6247a4f851c8967ab183de8cc875c"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.215372 30279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:18.243001 30279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:18.246604 30279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:18.246820 30279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:18.248456 30279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:18.248519 30279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:18.248564 30279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:18.248591 30279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:21:18 dist-test-slave-5bn2 krb5kdc[30075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486878, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:18.259361 30279 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:38385
I20260811 22:21:18.259392 30496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:38385 every 8 connection(s)
I20260811 22:21:18.260617 30279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:18.262669 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30279
I20260811 22:21:18.262786 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:18.263077 27124 external_mini_cluster.cc:1428] Setting key 46d37890e36f1756b8bbb93321fceaa9
I20260811 22:21:18.263230 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.249414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:53767 (local address 127.26.125.62:41713)
0811 22:21:18.249556 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.249560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.250838 (+ 1278us) server_negotiation.cc:407] Connection header received
0811 22:21:18.252047 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.252051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.252127 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.252227 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:18.253630 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.254458 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.255874 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.256253 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.256380 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.259102 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:18.259107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:18.259110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:18.259147 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:18.260824 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.261499 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.261503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.261504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.261611 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.261980 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.261982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.261983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.262200 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:18.262281 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.262887 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.263058 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:21:18.264135 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.250052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41713 (local address 127.26.125.1:53767)
0811 22:21:18.250667 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.250717 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.251773 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.252441 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.252452 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.252831 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:18.253480 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.253495 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.254585 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.254589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.255760 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.255771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.256443 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.256446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.256611 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.257231 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:18.257253 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:18.258824 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:18.260974 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.260979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.260995 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.261364 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.261710 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.261714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.261716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.261881 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.262282 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:18.262288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:18.262600 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.262773 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.263152 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":157,"threads_started":1}
I20260811 22:21:18.265403 30498 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41713
I20260811 22:21:18.265563 30498 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:18.266155 30498 heartbeater.cc:511] Master 127.26.125.62:41713 requested a full tablet report, sending...
I20260811 22:21:18.267776 30143 ts_manager.cc:194] Registered new tserver with Master: 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385)
I20260811 22:21:18.269145 30143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:53767
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:21:18.275890 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.267967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:48563 (local address 127.26.125.62:41713)
0811 22:21:18.268086 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.268091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.268402 (+ 311us) server_negotiation.cc:407] Connection header received
0811 22:21:18.268513 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.268516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.268563 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.268634 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:18.269119 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.269785 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.270916 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.271284 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.271407 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.272658 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:18.272662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:18.272665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:18.272692 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:18.274279 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.274705 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.274709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.274711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.274807 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.275129 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.275131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.275133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.275325 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:18.275410 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.275657 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.275765 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:21:18.275916 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.267888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41713 (local address 127.26.125.1:48563)
0811 22:21:18.268310 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.268325 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.268425 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.268662 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.268664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.268877 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:18.269000 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.269007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.269912 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.269915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.270754 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.270765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.271423 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.271427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.271526 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.272111 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:18.272124 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:18.272465 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:18.274399 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.274402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.274403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.274616 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.274945 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.274948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.274950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.275050 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.275406 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:18.275408 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:18.275501 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.275597 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.275753 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":231,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:18.293565 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41713
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:34731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:18.410827 30524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:18.411177 30524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:18.411247 30524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:18.415278 30524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:18.415354 30524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:18.415407 30524 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:21:18.415434 30524 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:18.415483 30524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:18.421074 30524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:34731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:41713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:18.422230 30524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:18.423339 30524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:18.433178 30539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:18.433139 30534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:18.433138 30532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:18.433207 30524 server_base.cc:1034] running on GCE node
I20260811 22:21:18.433836 30524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:18.434442 30524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:18.435606 30524 hybrid_clock.cc:648] HybridClock initialized: now 1786486878435584 us; error 34 us; skew 500 ppm
Aug 11 22:21:18 dist-test-slave-5bn2 krb5kdc[30075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486878, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:18.438301 30524 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:18.439186 30524 webserver.cc:460] Webserver started at http://127.26.125.2:40139/ using document root <none> and password file <none>
I20260811 22:21:18.439728 30524 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:18.439786 30524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:18.439958 30524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:18.441965 30524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "783534c1e1894ac4925eecde99ff22b5"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "9e646bff280bcb91b2f1d805406cff1a"
server_key_iv: "9c600ec8484b96ff050edd4ed3dab6ad"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.442480 30524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "783534c1e1894ac4925eecde99ff22b5"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "9e646bff280bcb91b2f1d805406cff1a"
server_key_iv: "9c600ec8484b96ff050edd4ed3dab6ad"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.446206 30524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:18.448731 30551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:18.449980 30524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 22:21:18.450102 30524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "783534c1e1894ac4925eecde99ff22b5"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "9e646bff280bcb91b2f1d805406cff1a"
server_key_iv: "9c600ec8484b96ff050edd4ed3dab6ad"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.450205 30524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:18.468264 30524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:18.471479 30524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:18.471673 30524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:18.473222 30524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:18.473278 30524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:18.473321 30524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:18.473349 30524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:18.484134 30524 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:46725
I20260811 22:21:18.484189 30750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:46725 every 8 connection(s)
I20260811 22:21:18.485105 30524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
Aug 11 22:21:18 dist-test-slave-5bn2 krb5kdc[30075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486878, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:18.491184 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30524
I20260811 22:21:18.491314 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:18.491586 27124 external_mini_cluster.cc:1428] Setting key b44e41d50221e1bb98dbf22f6a46d530
I20260811 22:21:18.493470 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.474017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:52875 (local address 127.26.125.62:41713)
0811 22:21:18.474139 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.474142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.475469 (+ 1327us) server_negotiation.cc:407] Connection header received
0811 22:21:18.476585 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.476589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.476652 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.476728 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:18.478289 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.478952 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.480788 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.481113 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.481234 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.489489 (+ 8255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:18.489492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:18.489495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:18.489544 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:18.491050 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.491660 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.491665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.491667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.491773 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.492163 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.492167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.492169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.492424 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:18.492529 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.493161 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.493303 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:21:18.494381 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.474708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41713 (local address 127.26.125.2:52875)
0811 22:21:18.475311 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.475361 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.476319 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.476929 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.476937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.477387 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:18.478110 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.478127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.479091 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.479095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.480608 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.480622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.481292 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.481296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.481507 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.487588 (+ 6081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:18.487608 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:18.489258 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:18.491190 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.491196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.491213 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.491543 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.491869 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.491871 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.491872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.492054 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.492540 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:18.492546 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:18.492859 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.493106 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.493504 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":155,"threads_started":1}
I20260811 22:21:18.495611 30757 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41713
I20260811 22:21:18.495777 30757 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:18.496316 30757 heartbeater.cc:511] Master 127.26.125.62:41713 requested a full tablet report, sending...
I20260811 22:21:18.497426 30143 ts_manager.cc:194] Registered new tserver with Master: 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725)
I20260811 22:21:18.498106 30143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:52875
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:18.505890 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.498175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:45767 (local address 127.26.125.62:41713)
0811 22:21:18.498288 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.498292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.498687 (+ 395us) server_negotiation.cc:407] Connection header received
0811 22:21:18.498822 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.498825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.498903 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.498983 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:18.499656 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.500443 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.501255 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.501608 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.501715 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.502890 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:18.502894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:18.502897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:18.502924 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:18.504236 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.504678 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.504682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.504684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.504768 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.505020 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.505023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.505025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.505258 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:18.505346 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.505612 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.505724 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:21:18.505905 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.498092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41713 (local address 127.26.125.2:45767)
0811 22:21:18.498599 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.498616 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.498729 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.499110 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.499113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.499335 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:18.499511 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.499521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.500580 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.500583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.501142 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.501151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.501742 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.501745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.501819 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.502366 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:18.502377 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:18.502729 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:18.504343 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.504346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.504348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.504585 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.504852 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.504855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.504857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.504941 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.505352 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:18.505355 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:18.505453 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.505539 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.505703 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:18.522594 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41713
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:34731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:18.644769 30772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:18.645076 30772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:18.645129 30772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:18.649394 30772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:18.649507 30772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:18.649554 30772 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:21:18.649586 30772 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:18.649641 30772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:18.655491 30772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:34731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:41713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:18.656687 30772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:18.657810 30772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:18.668097 30781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:18.668198 30772 server_base.cc:1034] running on GCE node
W20260811 22:21:18.668159 30787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:18.668171 30780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:18.668853 30772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:18.669539 30772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:18.670701 30772 hybrid_clock.cc:648] HybridClock initialized: now 1786486878670670 us; error 52 us; skew 500 ppm
Aug 11 22:21:18 dist-test-slave-5bn2 krb5kdc[30075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486878, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:18.673645 30772 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:18.674577 30772 webserver.cc:460] Webserver started at http://127.26.125.3:41999/ using document root <none> and password file <none>
I20260811 22:21:18.675182 30772 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:18.675247 30772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:18.675518 30772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:18.677675 30772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "593aa095192247e9816ed6f1ebaf230a"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "58b0f99944d045fe668785766beb0195"
server_key_iv: "31e1842651d72bdfca0d08c956d51d5d"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.678246 30772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "593aa095192247e9816ed6f1ebaf230a"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "58b0f99944d045fe668785766beb0195"
server_key_iv: "31e1842651d72bdfca0d08c956d51d5d"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.682224 30772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:18.685010 30799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:18.686462 30772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:18.686594 30772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "593aa095192247e9816ed6f1ebaf230a"
format_stamp: "Formatted at 2026-08-11 22:21:18 on dist-test-slave-5bn2"
server_key: "58b0f99944d045fe668785766beb0195"
server_key_iv: "31e1842651d72bdfca0d08c956d51d5d"
server_key_version: "encryptionkey@0"
I20260811 22:21:18.686709 30772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:18.699450 30772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:18.702944 30772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:18.703133 30772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:18.704689 30772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:18.704746 30772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:18.704803 30772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:18.704833 30772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:21:18 dist-test-slave-5bn2 krb5kdc[30075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486878, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:18.715296 30772 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:46029
I20260811 22:21:18.715368 31010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:46029 every 8 connection(s)
I20260811 22:21:18.716246 30772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:18.719370 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.705549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:48239 (local address 127.26.125.62:41713)
0811 22:21:18.705664 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.705668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.706823 (+ 1155us) server_negotiation.cc:407] Connection header received
0811 22:21:18.707958 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.707962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.708030 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.708124 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:18.709702 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.710524 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.712324 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.712546 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.712628 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.715779 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:18.715783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:18.715785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:18.715814 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:18.717335 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.717835 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.717838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.717839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.717934 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.718228 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.718230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.718231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.718475 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:18.718552 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.719123 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.719246 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:21:18.720229 30817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.706165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41713 (local address 127.26.125.3:48239)
0811 22:21:18.706673 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.706718 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.707693 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.708367 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.708378 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.708813 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:18.709534 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.709557 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.710674 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.710678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.712179 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.712190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.712721 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.712726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.712942 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.713681 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:18.713705 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:18.715604 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:18.717453 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.717458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.717470 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.717728 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.718010 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.718013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.718015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.718154 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.718575 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:18.718580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:18.718814 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.719046 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.719344 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":129,"threads_started":1}
I20260811 22:21:18.720732 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30772
I20260811 22:21:18.720845 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:18.721185 27124 external_mini_cluster.cc:1428] Setting key 729ad3b36efa6fd44cadaf5c41c12bbf
I20260811 22:21:18.721571 31011 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41713
I20260811 22:21:18.721714 31011 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:18.722277 31011 heartbeater.cc:511] Master 127.26.125.62:41713 requested a full tablet report, sending...
I20260811 22:21:18.723335 30146 ts_manager.cc:194] Registered new tserver with Master: 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029)
I20260811 22:21:18.724045 30146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:48239
I20260811 22:21:18.730254 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.723804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41713 (local address 127.26.125.3:36083)
0811 22:21:18.724362 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.724375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.724455 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.724659 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.724661 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.724825 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:18.724918 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.724923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.725898 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.725901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.726451 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.726459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.726892 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.726897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.726983 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.727393 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:18.727404 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:18.727751 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:18.729105 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.729108 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.729109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.729291 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.729510 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:18.729512 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:18.729513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:18.729582 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:18.729860 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:18.729862 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:18.729930 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.730001 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.730118 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":81,"threads_started":1}
I20260811 22:21:18.730261 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.723885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:36083 (local address 127.26.125.62:41713)
0811 22:21:18.724005 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.724009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.724440 (+ 431us) server_negotiation.cc:407] Connection header received
0811 22:21:18.724525 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.724528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.724575 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.724639 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:18.725029 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.725771 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.726562 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.726771 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.726845 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.727867 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:18.727870 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:18.727872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:18.727894 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:18.729009 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.729360 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.729363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.729364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.729428 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.729642 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.729644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.729645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.729800 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:18.729868 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.730046 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.730149 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:21:18.735013 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:21:18 dist-test-slave-5bn2 krb5kdc[30075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:21:18 dist-test-slave-5bn2 krb5kdc[30075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:18.762439 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.754580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33088 (local address 127.26.125.62:41713)
0811 22:21:18.754714 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.754717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.754731 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:18.754775 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.754777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.754817 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.754940 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:18.755679 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.756229 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.757063 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.757287 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.757433 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.759779 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:18.759782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:18.759784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:18.759812 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:18.761057 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.761424 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.761427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.761428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.761496 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:18.761680 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:18.761682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:18.761683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:18.761877 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:18.761972 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.762202 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.762306 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:21:18.765565 30146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:18.768736 30146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:18.783686 31051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.777570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46725 (local address 127.0.0.1:42564)
0811 22:21:18.778646 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.778661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.778750 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.779709 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.779712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.779732 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:18.779851 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.779859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.781140 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.781143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.782266 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.782275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.783349 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.783353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.783442 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.783463 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.783518 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.783570 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":48,"thread_start_us":103,"threads_started":1}
I20260811 22:21:18.783820 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.778385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42564 (local address 127.26.125.2:46725)
0811 22:21:18.779212 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.779223 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.779247 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:21:18.779323 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.779328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.779542 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.779734 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:18.779959 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.781020 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.782399 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.783233 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.783337 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.783500 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.783566 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.783667 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":125,"threads_started":1}
I20260811 22:21:18.784014 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.777602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.26.125.3:46029)
0811 22:21:18.778285 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.778298 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.778325 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:21:18.778405 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.778411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.778644 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.778819 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:18.779432 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.780476 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.782055 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.782951 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.783057 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.783249 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.783760 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.783866 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":114,"threads_started":1}
I20260811 22:21:18.783686 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.777475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:46029 (local address 127.0.0.1:38276)
0811 22:21:18.777886 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.777938 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.778094 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.778805 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.778812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.778833 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:18.779044 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.779053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.780579 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.780583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.781900 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.781911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.783061 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.783064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.783191 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.783235 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.783347 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.783507 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260811 22:21:18.784551 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.778308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:38385 (local address 127.0.0.1:38298)
0811 22:21:18.779704 (+ 1396us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.779718 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.779890 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.780402 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.780405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.780423 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:18.780514 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.780520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.781688 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.781692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.782887 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.782901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.784154 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.784158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.784257 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.784282 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.784348 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.784420 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"mutex_wait_us":48,"thread_start_us":100,"threads_started":1}
I20260811 22:21:18.784969 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.778468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.26.125.1:38385)
0811 22:21:18.778994 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.779001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.779817 (+ 816us) server_negotiation.cc:407] Connection header received
0811 22:21:18.780059 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.780064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.780270 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.780415 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:18.780669 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.781556 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.783037 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.783813 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.783923 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.784080 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.784727 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.784838 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":80,"threads_started":1}
I20260811 22:21:18.787274 30630 tablet_service.cc:1467] Processing CreateTablet for tablet e42670ca783e47ecb37fc7a160cfd48f (DEFAULT_TABLE table=test-table [id=a4816bded93142b0a6f6fc81f04788d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:18.787333 30892 tablet_service.cc:1467] Processing CreateTablet for tablet e42670ca783e47ecb37fc7a160cfd48f (DEFAULT_TABLE table=test-table [id=a4816bded93142b0a6f6fc81f04788d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:18.788302 30630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42670ca783e47ecb37fc7a160cfd48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:18.788319 30892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42670ca783e47ecb37fc7a160cfd48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:18.792968 30387 tablet_service.cc:1467] Processing CreateTablet for tablet e42670ca783e47ecb37fc7a160cfd48f (DEFAULT_TABLE table=test-table [id=a4816bded93142b0a6f6fc81f04788d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:18.794296 30387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42670ca783e47ecb37fc7a160cfd48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:18.796005 31057 tablet_bootstrap.cc:492] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Bootstrap starting.
I20260811 22:21:18.797453 31058 tablet_bootstrap.cc:492] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Bootstrap starting.
I20260811 22:21:18.798341 31057 tablet_bootstrap.cc:654] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:18.799182 31057 log.cc:824] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:18.799798 31058 tablet_bootstrap.cc:654] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:18.800714 31058 log.cc:824] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:18.801326 31057 tablet_bootstrap.cc:492] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: No bootstrap required, opened a new log
I20260811 22:21:18.801591 31057 ts_tablet_manager.cc:1397] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:21:18.802672 31065 tablet_bootstrap.cc:492] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Bootstrap starting.
I20260811 22:21:18.803049 31058 tablet_bootstrap.cc:492] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: No bootstrap required, opened a new log
I20260811 22:21:18.803258 31058 ts_tablet_manager.cc:1397] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:21:18.805279 31065 tablet_bootstrap.cc:654] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:18.805603 31057 raft_consensus.cc:348] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.805877 31057 raft_consensus.cc:374] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:18.805925 31057 raft_consensus.cc:729] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783534c1e1894ac4925eecde99ff22b5, State: Initialized, Role: FOLLOWER
I20260811 22:21:18.806164 31065 log.cc:824] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:18.806198 31058 raft_consensus.cc:348] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.806473 31058 raft_consensus.cc:374] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:18.806411 31057 consensus_queue.cc:260] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.806565 31058 raft_consensus.cc:729] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593aa095192247e9816ed6f1ebaf230a, State: Initialized, Role: FOLLOWER
I20260811 22:21:18.807060 31058 consensus_queue.cc:260] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.807132 30757 heartbeater.cc:503] Master 127.26.125.62:41713 was elected leader, sending a full tablet report...
I20260811 22:21:18.807519 31057 ts_tablet_manager.cc:1428] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:21:18.808120 31058 ts_tablet_manager.cc:1428] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:21:18.808450 31011 heartbeater.cc:503] Master 127.26.125.62:41713 was elected leader, sending a full tablet report...
I20260811 22:21:18.808647 31065 tablet_bootstrap.cc:492] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: No bootstrap required, opened a new log
I20260811 22:21:18.808833 31065 ts_tablet_manager.cc:1397] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:21:18.812958 31065 raft_consensus.cc:348] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.813277 31065 raft_consensus.cc:374] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:18.813344 31065 raft_consensus.cc:729] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9236254bb50f4d599bfb9c43830f21c3, State: Initialized, Role: FOLLOWER
I20260811 22:21:18.813753 31065 consensus_queue.cc:260] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.814374 30498 heartbeater.cc:503] Master 127.26.125.62:41713 was elected leader, sending a full tablet report...
I20260811 22:21:18.814652 31065 ts_tablet_manager.cc:1428] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:21:18.960134 31071 raft_consensus.cc:482] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:18.960384 31071 raft_consensus.cc:504] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.961616 31071 leader_election.cc:288] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029), 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385)
I20260811 22:21:18.966193 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.961993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:46029 (local address 127.26.125.2:53629)
0811 22:21:18.962153 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.962171 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.962294 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.962693 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.962697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.962716 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:18.962904 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.962912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.963852 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.963856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.964948 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.964959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.965732 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.965736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.965851 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.965875 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.965944 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.966029 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":77,"spinlock_wait_cycles":25856}
I20260811 22:21:18.966217 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.962072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:53629 (local address 127.26.125.3:46029)
0811 22:21:18.962208 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.962213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.962233 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:21:18.962433 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.962436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.962491 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.962564 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:18.963049 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.963717 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.965073 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.965617 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.965727 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.965788 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.965977 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.966082 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:21:18.966250 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.962509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:44447 (local address 127.26.125.1:38385)
0811 22:21:18.962648 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:18.962653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:18.962739 (+ 86us) server_negotiation.cc:407] Connection header received
0811 22:21:18.962892 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:18.962895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:18.962947 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:18.963020 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:18.963338 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.963872 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.965136 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.965638 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.965753 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.965807 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:18.966066 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 22:21:18.966147 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:21:18.966594 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:18.962293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:38385 (local address 127.26.125.2:44447)
0811 22:21:18.962649 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:18.962662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:18.962783 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:18.963055 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:18.963058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:18.963078 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:18.963213 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.963219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.964012 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.964015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.965005 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:18.965016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:18.965798 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:18.965804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:18.965908 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:18.965932 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:21:18.966366 (+ 434us) client_negotiation.cc:240] Negotiation successful
0811 22:21:18.966445 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":41,"spinlock_wait_cycles":384,"thread_start_us":139,"threads_started":1}
I20260811 22:21:18.967036 30932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42670ca783e47ecb37fc7a160cfd48f" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593aa095192247e9816ed6f1ebaf230a" is_pre_election: true
W20260811 22:21:18.967203 31014 tablet.cc:2369] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:18.967410 30932 raft_consensus.cc:2454] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 0.
I20260811 22:21:18.967895 30552 leader_election.cc:302] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 593aa095192247e9816ed6f1ebaf230a, 783534c1e1894ac4925eecde99ff22b5; no voters:
I20260811 22:21:18.968063 30422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42670ca783e47ecb37fc7a160cfd48f" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9236254bb50f4d599bfb9c43830f21c3" is_pre_election: true
I20260811 22:21:18.968235 31071 raft_consensus.cc:2781] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:18.968313 31071 raft_consensus.cc:482] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:18.968351 31071 raft_consensus.cc:3037] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:18.968382 30422 raft_consensus.cc:2454] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 0.
I20260811 22:21:18.969861 31071 raft_consensus.cc:504] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.970240 31071 leader_election.cc:288] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 election: Requested vote from peers 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029), 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385)
I20260811 22:21:18.970618 30932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42670ca783e47ecb37fc7a160cfd48f" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593aa095192247e9816ed6f1ebaf230a"
I20260811 22:21:18.970708 30422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42670ca783e47ecb37fc7a160cfd48f" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9236254bb50f4d599bfb9c43830f21c3"
I20260811 22:21:18.970775 30932 raft_consensus.cc:3037] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:18.970881 30422 raft_consensus.cc:3037] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:18.972011 30422 raft_consensus.cc:2454] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 1.
I20260811 22:21:18.972350 30552 leader_election.cc:302] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783534c1e1894ac4925eecde99ff22b5, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:18.972379 30932 raft_consensus.cc:2454] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 1.
I20260811 22:21:18.972550 31071 raft_consensus.cc:2781] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:18.972848 31071 raft_consensus.cc:686] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 1 LEADER]: Becoming Leader. State: Replica: 783534c1e1894ac4925eecde99ff22b5, State: Running, Role: LEADER
I20260811 22:21:18.973220 31071 consensus_queue.cc:237] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } }
I20260811 22:21:18.976747 30146 catalog_manager.cc:5495] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2). New cstate: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } health_report { overall_health: UNKNOWN } } }
W20260811 22:21:18.986349 30759 tablet.cc:2369] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:21:19.012195 30507 tablet.cc:2369] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:21:19 dist-test-slave-5bn2 krb5kdc[30075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486877, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:19.053503 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.044761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.26.125.62:41713)
0811 22:21:19.044910 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.044914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.045128 (+ 214us) server_negotiation.cc:407] Connection header received
0811 22:21:19.045255 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.045258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.045300 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.045371 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:19.046091 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.046615 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.047434 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.047652 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.047738 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.050285 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:19.050289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:19.050292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:19.050318 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:19.051672 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:19.052204 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:19.052208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:19.052210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:19.052308 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:19.052581 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:19.052584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:19.052587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:19.052777 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:19.052863 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.053204 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.053318 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":59}
I20260811 22:21:19.053627 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.044730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41713 (local address 127.0.0.1:33104)
0811 22:21:19.045039 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:19.045052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:19.045173 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:19.045403 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:19.045406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:19.045870 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:19.045968 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.045975 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.046721 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.046724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.047324 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.047333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.047752 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.047756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.047870 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.048508 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:19.048520 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:19.050140 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:19.051789 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:19.051802 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:19.051810 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:19.052116 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:19.052395 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:19.052398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:19.052400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:19.052484 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:19.052879 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:19.052888 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:19.053017 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 22:21:19.053154 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 22:21:19.053458 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":103,"thread_start_us":121,"threads_started":1}
I20260811 22:21:19.058172 30146 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:33104:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:21:19.058954 30146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:19.064368 30630 tablet_service.cc:1467] Processing CreateTablet for tablet 62b0d14f1f4145058b8b499af3a2f29b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a44a970fc0bc488d9f4944164a568164]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:19.064368 30387 tablet_service.cc:1467] Processing CreateTablet for tablet 62b0d14f1f4145058b8b499af3a2f29b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a44a970fc0bc488d9f4944164a568164]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:19.065246 30387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b0d14f1f4145058b8b499af3a2f29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:19.065246 30630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b0d14f1f4145058b8b499af3a2f29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:19.064747 30892 tablet_service.cc:1467] Processing CreateTablet for tablet 62b0d14f1f4145058b8b499af3a2f29b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a44a970fc0bc488d9f4944164a568164]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:19.065524 30892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b0d14f1f4145058b8b499af3a2f29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:19.068324 31058 tablet_bootstrap.cc:492] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: Bootstrap starting.
I20260811 22:21:19.069298 31058 tablet_bootstrap.cc:654] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:19.069880 31065 tablet_bootstrap.cc:492] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: Bootstrap starting.
I20260811 22:21:19.070618 31058 tablet_bootstrap.cc:492] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: No bootstrap required, opened a new log
I20260811 22:21:19.070701 31058 ts_tablet_manager.cc:1397] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:19.070919 31057 tablet_bootstrap.cc:492] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: Bootstrap starting.
I20260811 22:21:19.070942 31065 tablet_bootstrap.cc:654] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:19.071247 31058 raft_consensus.cc:348] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.071367 31058 raft_consensus.cc:374] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:19.071398 31058 raft_consensus.cc:729] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593aa095192247e9816ed6f1ebaf230a, State: Initialized, Role: FOLLOWER
I20260811 22:21:19.071542 31058 consensus_queue.cc:260] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.071839 31072 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.071996 31057 tablet_bootstrap.cc:654] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:19.072017 31072 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:19.071826 31058 ts_tablet_manager.cc:1428] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:19.072284 31065 tablet_bootstrap.cc:492] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: No bootstrap required, opened a new log
I20260811 22:21:19.072368 31065 ts_tablet_manager.cc:1397] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:21:19.072835 31065 raft_consensus.cc:348] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.072966 31065 raft_consensus.cc:374] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:19.073005 31065 raft_consensus.cc:729] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9236254bb50f4d599bfb9c43830f21c3, State: Initialized, Role: FOLLOWER
I20260811 22:21:19.073136 31065 consensus_queue.cc:260] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.073413 31065 ts_tablet_manager.cc:1428] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:21:19.073531 31057 tablet_bootstrap.cc:492] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: No bootstrap required, opened a new log
I20260811 22:21:19.073627 31057 ts_tablet_manager.cc:1397] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:19.073678 31075 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.073827 31075 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:19.074064 31057 raft_consensus.cc:348] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.074169 31057 raft_consensus.cc:374] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:19.074209 31057 raft_consensus.cc:729] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783534c1e1894ac4925eecde99ff22b5, State: Initialized, Role: FOLLOWER
I20260811 22:21:19.074337 31057 consensus_queue.cc:260] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.074587 31057 ts_tablet_manager.cc:1428] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:21:19.074643 31071 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.074797 31071 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:19.407488 31071 raft_consensus.cc:482] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:19.407672 31071 raft_consensus.cc:504] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.408093 31071 leader_election.cc:288] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385), 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029)
I20260811 22:21:19.408622 30422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b0d14f1f4145058b8b499af3a2f29b" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9236254bb50f4d599bfb9c43830f21c3" is_pre_election: true
I20260811 22:21:19.408775 30422 raft_consensus.cc:2454] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 0.
I20260811 22:21:19.408713 30932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b0d14f1f4145058b8b499af3a2f29b" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593aa095192247e9816ed6f1ebaf230a" is_pre_election: true
I20260811 22:21:19.408885 30932 raft_consensus.cc:2454] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 0.
I20260811 22:21:19.409090 30552 leader_election.cc:302] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783534c1e1894ac4925eecde99ff22b5, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:19.409276 31071 raft_consensus.cc:2781] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:19.409345 31071 raft_consensus.cc:482] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:19.409394 31071 raft_consensus.cc:3037] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:19.410408 31071 raft_consensus.cc:504] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.410744 31071 leader_election.cc:288] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 election: Requested vote from peers 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385), 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029)
I20260811 22:21:19.411181 30422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b0d14f1f4145058b8b499af3a2f29b" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9236254bb50f4d599bfb9c43830f21c3"
I20260811 22:21:19.411214 30932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b0d14f1f4145058b8b499af3a2f29b" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593aa095192247e9816ed6f1ebaf230a"
I20260811 22:21:19.411326 30932 raft_consensus.cc:3037] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:19.411326 30422 raft_consensus.cc:3037] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:19.412503 30422 raft_consensus.cc:2454] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 1.
I20260811 22:21:19.412505 30932 raft_consensus.cc:2454] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 1.
I20260811 22:21:19.412843 30552 leader_election.cc:302] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783534c1e1894ac4925eecde99ff22b5, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:19.413023 31071 raft_consensus.cc:2781] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:19.413152 31071 raft_consensus.cc:686] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [term 1 LEADER]: Becoming Leader. State: Replica: 783534c1e1894ac4925eecde99ff22b5, State: Running, Role: LEADER
I20260811 22:21:19.413297 31071 consensus_queue.cc:237] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.413950 31077 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: TxnStatusTablet state changed. Reason: New leader 783534c1e1894ac4925eecde99ff22b5. Latest consensus state: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.414077 31077 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:19.414348 31100 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:21:19.414572 31100 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:21:19.415164 30144 catalog_manager.cc:5495] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2). New cstate: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } health_report { overall_health: UNKNOWN } } }
I20260811 22:21:19.443048 31077 consensus_queue.cc:1035] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:19.443842 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.439302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.26.125.2:46725)
0811 22:21:19.439628 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.439637 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.439654 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:21:19.439741 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.439744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.439815 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.439933 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:19.440180 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.441061 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.442063 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.442314 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.442406 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.442832 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.443419 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.443552 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.443646 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.443699 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":132,"threads_started":1}
I20260811 22:21:19.443884 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.439207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46725 (local address 127.0.0.1:42566)
0811 22:21:19.439388 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:19.439406 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:19.439525 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:19.439945 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:19.439948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:19.439955 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:19.440063 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.440068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.441181 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.441184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.441933 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.441945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.442417 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.442421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.442511 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.442566 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.443527 (+ 961us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.443553 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:21:19.443635 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:21:19.443735 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 22:21:19.449836 30420 raft_consensus.cc:1261] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Refusing update from remote peer 783534c1e1894ac4925eecde99ff22b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:19.450271 30927 raft_consensus.cc:1261] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Refusing update from remote peer 783534c1e1894ac4925eecde99ff22b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:19.450770 31077 consensus_queue.cc:1035] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:19.451303 31071 consensus_queue.cc:1035] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:19.451574 31071 consensus_queue.cc:1035] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:19.453677 31075 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: TxnStatusTablet state changed. Reason: New leader 783534c1e1894ac4925eecde99ff22b5. Latest consensus state: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.453814 31075 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:19.457484 31072 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: TxnStatusTablet state changed. Reason: New leader 783534c1e1894ac4925eecde99ff22b5. Latest consensus state: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.457624 31072 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:19.458455 31077 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.458621 31077 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:19.458746 31105 mvcc.cc:205] Tried to move back new op lower bound from 7317450258220376064 to 7317450258078138368. Current Snapshot: MvccSnapshot[applied={T|T < 7317450258220376064}]
I20260811 22:21:19.459543 31075 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.459663 31075 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 9236254bb50f4d599bfb9c43830f21c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:19.464089 31071 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.464236 31071 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 783534c1e1894ac4925eecde99ff22b5: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:19.466917 31113 tablet_replica.cc:439] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } } }
I20260811 22:21:19.467075 31113 tablet_replica.cc:442] T 62b0d14f1f4145058b8b499af3a2f29b P 593aa095192247e9816ed6f1ebaf230a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:19.467760 31121 mvcc.cc:205] Tried to move back new op lower bound from 7317450258220376064 to 7317450258078138368. Current Snapshot: MvccSnapshot[applied={T|T < 7317450258220376064}]
I20260811 22:21:19.474618 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.470712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.26.125.2:46725)
0811 22:21:19.470834 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.470837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.471176 (+ 339us) server_negotiation.cc:407] Connection header received
0811 22:21:19.471317 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.471319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.471364 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.471418 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:19.471672 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.472203 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.473433 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.473672 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.473755 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.474063 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.474163 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.474234 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.474347 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.474403 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:21:19.487149 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.482907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:35061 (local address 127.26.125.2:46725)
0811 22:21:19.483046 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.483050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.483275 (+ 225us) server_negotiation.cc:407] Connection header received
0811 22:21:19.483400 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.483402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.483446 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.483521 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:19.483836 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.484547 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.485786 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.486122 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.486247 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.486603 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.486698 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.486777 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.486889 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.486954 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":55}
I20260811 22:21:19.487177 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.482807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46725 (local address 127.26.125.2:35061)
0811 22:21:19.483189 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:19.483203 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:19.483318 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:19.483535 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:19.483538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:19.483546 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:19.483704 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.483709 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.484681 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.484684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.485647 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.485659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.486265 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.486268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.486365 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.486428 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.486787 (+ 359us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.486793 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:21:19.486897 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:21:19.486982 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":15,"thread_start_us":172,"threads_started":1}
I20260811 22:21:19.534341 30143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:19.534571 30143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:19.537197 30143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: Sending DeleteTablet for 3 replicas of tablet e42670ca783e47ecb37fc7a160cfd48f
I20260811 22:21:19.538002 30630 tablet_service.cc:1514] Processing DeleteTablet for tablet e42670ca783e47ecb37fc7a160cfd48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:19 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:42564
I20260811 22:21:19.538493 30892 tablet_service.cc:1514] Processing DeleteTablet for tablet e42670ca783e47ecb37fc7a160cfd48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:19 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:38276
I20260811 22:21:19.538587 30387 tablet_service.cc:1514] Processing DeleteTablet for tablet e42670ca783e47ecb37fc7a160cfd48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:19 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:38298
I20260811 22:21:19.539059 31146 tablet_replica.cc:330] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: stopping tablet replica
I20260811 22:21:19.539130 31144 tablet_replica.cc:330] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: stopping tablet replica
I20260811 22:21:19.539096 30143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:21:19.539521 31144 raft_consensus.cc:2229] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 1 LEADER]: Raft consensus shutting down.
W20260811 22:21:19.539543 30143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:19.539572 31146 raft_consensus.cc:2229] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:19.539928 31146 raft_consensus.cc:2258] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:19.539927 31144 raft_consensus.cc:2258] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:19.541384 31144 ts_tablet_manager.cc:1905] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:19.541384 31146 ts_tablet_manager.cc:1905] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:19.544714 31146 ts_tablet_manager.cc:1918] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:19.544936 31146 log.cc:1196] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/e42670ca783e47ecb37fc7a160cfd48f
I20260811 22:21:19.545323 30630 tablet_service.cc:1467] Processing CreateTablet for tablet 949681c948134e808bd68c914a99a248 (DEFAULT_TABLE table=test-table [id=629dd06c70304afd99f7148366b360cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:19.545536 31146 ts_tablet_manager.cc:1939] T e42670ca783e47ecb37fc7a160cfd48f P 593aa095192247e9816ed6f1ebaf230a: Deleting consensus metadata
I20260811 22:21:19.545640 30630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 949681c948134e808bd68c914a99a248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:19.545847 30387 tablet_service.cc:1467] Processing CreateTablet for tablet 949681c948134e808bd68c914a99a248 (DEFAULT_TABLE table=test-table [id=629dd06c70304afd99f7148366b360cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:19.546159 30387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 949681c948134e808bd68c914a99a248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:19.547886 31057 tablet_bootstrap.cc:492] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: Bootstrap starting.
I20260811 22:21:19.548199 31144 ts_tablet_manager.cc:1918] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:19.548309 31144 log.cc:1196] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/e42670ca783e47ecb37fc7a160cfd48f
I20260811 22:21:19.548619 31065 tablet_bootstrap.cc:492] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: Bootstrap starting.
I20260811 22:21:19.548650 31144 ts_tablet_manager.cc:1939] T e42670ca783e47ecb37fc7a160cfd48f P 783534c1e1894ac4925eecde99ff22b5: Deleting consensus metadata
I20260811 22:21:19.548834 30125 catalog_manager.cc:4841] TS 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029): tablet e42670ca783e47ecb37fc7a160cfd48f (table test-table [id=a4816bded93142b0a6f6fc81f04788d9]) successfully deleted
I20260811 22:21:19.548909 31057 tablet_bootstrap.cc:654] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:19.549597 31065 tablet_bootstrap.cc:654] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:19.549604 30892 tablet_service.cc:1467] Processing CreateTablet for tablet 949681c948134e808bd68c914a99a248 (DEFAULT_TABLE table=test-table [id=629dd06c70304afd99f7148366b360cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:19.549885 30892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 949681c948134e808bd68c914a99a248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:19.549929 30127 catalog_manager.cc:4841] TS 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725): tablet e42670ca783e47ecb37fc7a160cfd48f (table test-table [id=a4816bded93142b0a6f6fc81f04788d9]) successfully deleted
I20260811 22:21:19.550489 31057 tablet_bootstrap.cc:492] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: No bootstrap required, opened a new log
I20260811 22:21:19.550575 31057 ts_tablet_manager.cc:1397] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:19.551061 31057 raft_consensus.cc:348] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.551175 31057 raft_consensus.cc:374] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:19.551205 31057 raft_consensus.cc:729] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783534c1e1894ac4925eecde99ff22b5, State: Initialized, Role: FOLLOWER
I20260811 22:21:19.551332 31057 consensus_queue.cc:260] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.551653 31057 ts_tablet_manager.cc:1428] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:19.552201 31149 tablet_replica.cc:330] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: stopping tablet replica
I20260811 22:21:19.552488 31058 tablet_bootstrap.cc:492] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: Bootstrap starting.
I20260811 22:21:19.553481 31058 tablet_bootstrap.cc:654] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:19.553408 31149 raft_consensus.cc:2229] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:19.553750 31149 raft_consensus.cc:2258] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:19.555613 31149 ts_tablet_manager.cc:1905] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:19.555656 31058 tablet_bootstrap.cc:492] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: No bootstrap required, opened a new log
I20260811 22:21:19.555744 31058 ts_tablet_manager.cc:1397] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:19.556300 31058 raft_consensus.cc:348] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.556435 31058 raft_consensus.cc:374] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:19.556475 31058 raft_consensus.cc:729] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593aa095192247e9816ed6f1ebaf230a, State: Initialized, Role: FOLLOWER
I20260811 22:21:19.556603 31058 consensus_queue.cc:260] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.557694 31058 ts_tablet_manager.cc:1428] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:19.558730 31149 ts_tablet_manager.cc:1918] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:19.558876 31149 log.cc:1196] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/e42670ca783e47ecb37fc7a160cfd48f
I20260811 22:21:19.559208 31149 ts_tablet_manager.cc:1939] T e42670ca783e47ecb37fc7a160cfd48f P 9236254bb50f4d599bfb9c43830f21c3: Deleting consensus metadata
I20260811 22:21:19.560119 31065 tablet_bootstrap.cc:492] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: No bootstrap required, opened a new log
I20260811 22:21:19.560220 31065 ts_tablet_manager.cc:1397] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20260811 22:21:19.560534 30125 catalog_manager.cc:4841] TS 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385): tablet e42670ca783e47ecb37fc7a160cfd48f (table test-table [id=a4816bded93142b0a6f6fc81f04788d9]) successfully deleted
I20260811 22:21:19.560688 31065 raft_consensus.cc:348] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.560797 31065 raft_consensus.cc:374] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:19.560827 31065 raft_consensus.cc:729] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9236254bb50f4d599bfb9c43830f21c3, State: Initialized, Role: FOLLOWER
I20260811 22:21:19.560951 31065 consensus_queue.cc:260] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.561183 31065 ts_tablet_manager.cc:1428] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:21:19.623373 31075 raft_consensus.cc:482] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:19.623540 31075 raft_consensus.cc:504] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.624858 31075 leader_election.cc:288] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725), 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029)
I20260811 22:21:19.629648 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.625249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:36693 (local address 127.26.125.2:46725)
0811 22:21:19.625372 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.625377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.625494 (+ 117us) server_negotiation.cc:407] Connection header received
0811 22:21:19.625724 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.625728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.625779 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.625826 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:19.626302 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.627095 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.628457 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.629098 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.629221 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.629281 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.629450 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.629534 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:21:19.629670 31158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.625077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46725 (local address 127.26.125.1:36693)
0811 22:21:19.625398 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:19.625412 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:19.625587 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:19.625974 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:19.625977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:19.626021 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:19.626143 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.626148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.627229 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.627233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.628337 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.628349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.629236 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.629239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.629320 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.629343 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:21:19.629409 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:21:19.629489 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":6,"spinlock_wait_cycles":1408,"thread_start_us":80,"threads_started":1}
I20260811 22:21:19.630203 30661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "949681c948134e808bd68c914a99a248" candidate_uuid: "9236254bb50f4d599bfb9c43830f21c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783534c1e1894ac4925eecde99ff22b5" is_pre_election: true
I20260811 22:21:19.630575 30661 raft_consensus.cc:2454] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9236254bb50f4d599bfb9c43830f21c3 in term 0.
I20260811 22:21:19.631181 31156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.625077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:46029 (local address 127.26.125.1:56251)
0811 22:21:19.625572 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:19.625586 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:19.625696 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:19.625939 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:19.625942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:19.625957 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:19.626101 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.626108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.626974 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.626977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.628338 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.628349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.630656 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.630660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.630805 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.630831 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:21:19.630933 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:21:19.631004 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":154,"thread_start_us":63,"threads_started":1}
I20260811 22:21:19.631211 31159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.625419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:56251 (local address 127.26.125.3:46029)
0811 22:21:19.625682 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.625692 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.625706 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:19.625785 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.625787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.625843 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.625921 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:19.626227 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.626823 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.629342 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.630341 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.630456 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.630506 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.630980 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.631070 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":64,"threads_started":1}
I20260811 22:21:19.631294 30307 leader_election.cc:302] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783534c1e1894ac4925eecde99ff22b5, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:19.631600 31075 raft_consensus.cc:2781] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:19.631661 31075 raft_consensus.cc:482] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:19.631701 31075 raft_consensus.cc:3037] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:19.631716 30932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "949681c948134e808bd68c914a99a248" candidate_uuid: "9236254bb50f4d599bfb9c43830f21c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593aa095192247e9816ed6f1ebaf230a" is_pre_election: true
I20260811 22:21:19.631853 30932 raft_consensus.cc:2454] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9236254bb50f4d599bfb9c43830f21c3 in term 0.
I20260811 22:21:19.632802 31075 raft_consensus.cc:504] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.633175 31075 leader_election.cc:288] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [CANDIDATE]: Term 1 election: Requested vote from peers 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725), 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029)
I20260811 22:21:19.633540 30932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "949681c948134e808bd68c914a99a248" candidate_uuid: "9236254bb50f4d599bfb9c43830f21c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593aa095192247e9816ed6f1ebaf230a"
I20260811 22:21:19.633665 30932 raft_consensus.cc:3037] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:19.634821 30932 raft_consensus.cc:2454] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9236254bb50f4d599bfb9c43830f21c3 in term 1.
I20260811 22:21:19.635435 30304 leader_election.cc:302] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 593aa095192247e9816ed6f1ebaf230a, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:19.635618 31075 raft_consensus.cc:2781] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:19.635855 31075 raft_consensus.cc:686] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9236254bb50f4d599bfb9c43830f21c3, State: Running, Role: LEADER
I20260811 22:21:19.635111 30661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "949681c948134e808bd68c914a99a248" candidate_uuid: "9236254bb50f4d599bfb9c43830f21c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783534c1e1894ac4925eecde99ff22b5"
I20260811 22:21:19.635991 30661 raft_consensus.cc:3037] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:19.636147 31075 consensus_queue.cc:237] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:19.637200 30661 raft_consensus.cc:2454] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9236254bb50f4d599bfb9c43830f21c3 in term 1.
I20260811 22:21:19.639063 30143 catalog_manager.cc:5495] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1). New cstate: current_term: 1 leader_uuid: "9236254bb50f4d599bfb9c43830f21c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } health_report { overall_health: UNKNOWN } } }
I20260811 22:21:19.672335 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.668595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38306 (local address 127.26.125.1:38385)
0811 22:21:19.668931 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.668938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.668949 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:21:19.669021 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.669032 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.669086 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.669197 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:19.669433 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.669981 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.670806 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.671103 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.671194 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.671456 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.671932 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.672049 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.672108 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.672166 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":157,"threads_started":1}
I20260811 22:21:19.683997 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.680446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:49381 (local address 127.26.125.2:46725)
0811 22:21:19.680578 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.680582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.680727 (+ 145us) server_negotiation.cc:407] Connection header received
0811 22:21:19.680864 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.680866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.680909 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.680969 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:19.681252 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.681767 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.683022 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.683235 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.683306 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.683630 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.683701 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.683746 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.683837 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.683883 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:21:19.684087 31172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.680289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46725 (local address 127.26.125.1:49381)
0811 22:21:19.680638 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:19.680654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:19.680774 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:19.680997 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:19.681000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:19.681008 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:19.681160 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.681167 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.681870 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.681873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.682919 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.682929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.683335 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.683339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.683440 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.683503 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.683766 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.683771 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:21:19.683827 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 22:21:19.683920 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":115,"threads_started":1}
I20260811 22:21:19.690991 30661 raft_consensus.cc:1261] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Refusing update from remote peer 9236254bb50f4d599bfb9c43830f21c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:19.691519 30932 raft_consensus.cc:1261] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Refusing update from remote peer 9236254bb50f4d599bfb9c43830f21c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:19.691671 31075 consensus_queue.cc:1035] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:19.692373 31075 consensus_queue.cc:1035] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:19.696115 31108 mvcc.cc:205] Tried to move back new op lower bound from 7317450259208380416 to 7317450258993000448. Current Snapshot: MvccSnapshot[applied={T|T < 7317450259208380416}]
I20260811 22:21:19.711903 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.707610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:38385 (local address 127.26.125.2:57405)
0811 22:21:19.707867 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:19.707883 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:19.707978 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:19.708533 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:19.708536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:19.708545 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:19.708644 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.708651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.709687 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.709691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.710618 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:19.710631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.711152 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.711156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:19.711246 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.711274 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.711590 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.711597 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:21:19.711664 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:21:19.711757 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260811 22:21:19.711903 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:19.708119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:57405 (local address 127.26.125.1:38385)
0811 22:21:19.708232 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:19.708236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:19.708250 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:19.708288 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:19.708291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:19.708339 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:19.708410 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:19.708861 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.709559 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.710740 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:19.711047 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:19.711142 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:19.711397 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:19.711488 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:19.711551 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:19.711707 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 22:21:19.711772 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 22:21:19.712750 31108 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:21:21.273378 30143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:21.273561 30143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:21.276244 30143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: Sending DeleteTablet for 3 replicas of tablet 949681c948134e808bd68c914a99a248
I20260811 22:21:21.276757 30387 tablet_service.cc:1514] Processing DeleteTablet for tablet 949681c948134e808bd68c914a99a248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:21 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:38298
I20260811 22:21:21.276791 30630 tablet_service.cc:1514] Processing DeleteTablet for tablet 949681c948134e808bd68c914a99a248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:21 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:42564
I20260811 22:21:21.276906 30892 tablet_service.cc:1514] Processing DeleteTablet for tablet 949681c948134e808bd68c914a99a248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:21 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:38276
I20260811 22:21:21.277137 31220 tablet_replica.cc:330] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: stopping tablet replica
I20260811 22:21:21.277267 31222 tablet_replica.cc:330] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: stopping tablet replica
I20260811 22:21:21.277274 31220 raft_consensus.cc:2229] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:21.277452 31222 raft_consensus.cc:2229] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:21.277586 31222 raft_consensus.cc:2258] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:21.277602 31220 raft_consensus.cc:2258] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:21.277880 31221 tablet_replica.cc:330] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: stopping tablet replica
I20260811 22:21:21.277987 31221 raft_consensus.cc:2229] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:21.278084 31221 raft_consensus.cc:2258] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:21.278473 31222 ts_tablet_manager.cc:1905] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:21.278570 31220 ts_tablet_manager.cc:1905] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:21.279261 31221 ts_tablet_manager.cc:1905] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:21.281152 31222 ts_tablet_manager.cc:1918] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:21:21.281284 31222 log.cc:1196] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/949681c948134e808bd68c914a99a248
I20260811 22:21:21.281610 31220 ts_tablet_manager.cc:1918] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:21:21.281635 31222 ts_tablet_manager.cc:1939] T 949681c948134e808bd68c914a99a248 P 593aa095192247e9816ed6f1ebaf230a: Deleting consensus metadata
I20260811 22:21:21.281738 31220 log.cc:1196] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/949681c948134e808bd68c914a99a248
I20260811 22:21:21.281967 31221 ts_tablet_manager.cc:1918] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:21:21.282043 31220 ts_tablet_manager.cc:1939] T 949681c948134e808bd68c914a99a248 P 9236254bb50f4d599bfb9c43830f21c3: Deleting consensus metadata
I20260811 22:21:21.282096 31221 log.cc:1196] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/949681c948134e808bd68c914a99a248
I20260811 22:21:21.282388 31221 ts_tablet_manager.cc:1939] T 949681c948134e808bd68c914a99a248 P 783534c1e1894ac4925eecde99ff22b5: Deleting consensus metadata
I20260811 22:21:21.282603 30125 catalog_manager.cc:4841] TS 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029): tablet 949681c948134e808bd68c914a99a248 (table test-table [id=629dd06c70304afd99f7148366b360cb]) successfully deleted
I20260811 22:21:21.282950 30125 catalog_manager.cc:4841] TS 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385): tablet 949681c948134e808bd68c914a99a248 (table test-table [id=629dd06c70304afd99f7148366b360cb]) successfully deleted
I20260811 22:21:21.283080 30127 catalog_manager.cc:4841] TS 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725): tablet 949681c948134e808bd68c914a99a248 (table test-table [id=629dd06c70304afd99f7148366b360cb]) successfully deleted
Aug 11 22:21:21 dist-test-slave-5bn2 krb5kdc[30075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:21:21 dist-test-slave-5bn2 krb5kdc[30075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:21.307550 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.298559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.26.125.62:41713)
0811 22:21:21.298872 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:21.298881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:21.298902 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:21:21.299026 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:21.299029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:21.299078 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:21.299161 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:21.299865 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.300433 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.301319 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.301585 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.301665 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.304273 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:21.304277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:21.304279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:21.304307 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:21.305846 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:21.306262 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:21.306265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:21.306266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:21.306342 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:21.306544 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:21.306547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:21.306548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:21.306736 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:21.306839 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:21.307241 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 22:21:21.307410 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":101,"threads_started":1}
I20260811 22:21:21.310364 30143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:21.310791 30143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:21.315718 30630 tablet_service.cc:1467] Processing CreateTablet for tablet c48478a655d845ec8f1282d5f7c6bd08 (DEFAULT_TABLE table=test-table [id=23ad082b9bdc43f5a1910d3e5ae3e742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:21.315685 30387 tablet_service.cc:1467] Processing CreateTablet for tablet c48478a655d845ec8f1282d5f7c6bd08 (DEFAULT_TABLE table=test-table [id=23ad082b9bdc43f5a1910d3e5ae3e742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:21.315719 30892 tablet_service.cc:1467] Processing CreateTablet for tablet c48478a655d845ec8f1282d5f7c6bd08 (DEFAULT_TABLE table=test-table [id=23ad082b9bdc43f5a1910d3e5ae3e742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:21.316020 30630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48478a655d845ec8f1282d5f7c6bd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:21.316020 30387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48478a655d845ec8f1282d5f7c6bd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:21.316030 30892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48478a655d845ec8f1282d5f7c6bd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:21.318239 31240 tablet_bootstrap.cc:492] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: Bootstrap starting.
I20260811 22:21:21.318584 31241 tablet_bootstrap.cc:492] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: Bootstrap starting.
I20260811 22:21:21.318991 31242 tablet_bootstrap.cc:492] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: Bootstrap starting.
I20260811 22:21:21.319274 31240 tablet_bootstrap.cc:654] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:21.319451 31241 tablet_bootstrap.cc:654] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:21.319996 31242 tablet_bootstrap.cc:654] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:21.320700 31240 tablet_bootstrap.cc:492] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: No bootstrap required, opened a new log
I20260811 22:21:21.320799 31240 ts_tablet_manager.cc:1397] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:21.320957 31241 tablet_bootstrap.cc:492] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: No bootstrap required, opened a new log
I20260811 22:21:21.321043 31241 ts_tablet_manager.cc:1397] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:21.321085 31242 tablet_bootstrap.cc:492] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: No bootstrap required, opened a new log
I20260811 22:21:21.321211 31242 ts_tablet_manager.cc:1397] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:21.321277 31240 raft_consensus.cc:348] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.321399 31240 raft_consensus.cc:374] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:21.321434 31240 raft_consensus.cc:729] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593aa095192247e9816ed6f1ebaf230a, State: Initialized, Role: FOLLOWER
I20260811 22:21:21.321395 31241 raft_consensus.cc:348] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.321480 31241 raft_consensus.cc:374] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:21.321501 31241 raft_consensus.cc:729] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9236254bb50f4d599bfb9c43830f21c3, State: Initialized, Role: FOLLOWER
I20260811 22:21:21.321569 31240 consensus_queue.cc:260] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.321621 31241 consensus_queue.cc:260] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.321887 31241 ts_tablet_manager.cc:1428] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:21:21.321938 31240 ts_tablet_manager.cc:1428] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:21:21.322641 31242 raft_consensus.cc:348] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.322911 31242 raft_consensus.cc:374] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:21.322973 31242 raft_consensus.cc:729] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783534c1e1894ac4925eecde99ff22b5, State: Initialized, Role: FOLLOWER
I20260811 22:21:21.323127 31242 consensus_queue.cc:260] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.323405 31242 ts_tablet_manager.cc:1428] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 22:21:21.485018 31179 raft_consensus.cc:482] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:21.485157 31179 raft_consensus.cc:504] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.485761 31179 leader_election.cc:288] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385), 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029)
I20260811 22:21:21.486202 30422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48478a655d845ec8f1282d5f7c6bd08" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9236254bb50f4d599bfb9c43830f21c3" is_pre_election: true
I20260811 22:21:21.486263 30927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48478a655d845ec8f1282d5f7c6bd08" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593aa095192247e9816ed6f1ebaf230a" is_pre_election: true
I20260811 22:21:21.486351 30422 raft_consensus.cc:2454] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 0.
I20260811 22:21:21.486414 30927 raft_consensus.cc:2454] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 0.
I20260811 22:21:21.486681 30552 leader_election.cc:302] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783534c1e1894ac4925eecde99ff22b5, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:21.486887 31179 raft_consensus.cc:2781] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:21.486960 31179 raft_consensus.cc:482] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:21.487006 31179 raft_consensus.cc:3037] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:21.488112 31179 raft_consensus.cc:504] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.488456 31179 leader_election.cc:288] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 election: Requested vote from peers 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385), 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029)
I20260811 22:21:21.488884 30422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48478a655d845ec8f1282d5f7c6bd08" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9236254bb50f4d599bfb9c43830f21c3"
I20260811 22:21:21.488900 30927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48478a655d845ec8f1282d5f7c6bd08" candidate_uuid: "783534c1e1894ac4925eecde99ff22b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593aa095192247e9816ed6f1ebaf230a"
I20260811 22:21:21.489022 30422 raft_consensus.cc:3037] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:21.489024 30927 raft_consensus.cc:3037] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:21.490202 30927 raft_consensus.cc:2454] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 1.
I20260811 22:21:21.490224 30422 raft_consensus.cc:2454] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783534c1e1894ac4925eecde99ff22b5 in term 1.
I20260811 22:21:21.490586 30552 leader_election.cc:302] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783534c1e1894ac4925eecde99ff22b5, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:21.490763 31179 raft_consensus.cc:2781] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:21.490919 31179 raft_consensus.cc:686] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 1 LEADER]: Becoming Leader. State: Replica: 783534c1e1894ac4925eecde99ff22b5, State: Running, Role: LEADER
I20260811 22:21:21.491068 31179 consensus_queue.cc:237] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.492990 30146 catalog_manager.cc:5495] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2). New cstate: current_term: 1 leader_uuid: "783534c1e1894ac4925eecde99ff22b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } health_report { overall_health: UNKNOWN } } }
I20260811 22:21:21.522274 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.519090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.26.125.2:46725)
0811 22:21:21.519419 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:21.519427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:21.519439 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:21:21.519478 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:21.519480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:21.519523 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:21.519592 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:21.519810 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.520335 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.521183 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.521506 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.521586 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.521833 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:21.521911 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:21.521979 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:21.522066 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 22:21:21.522114 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":129,"threads_started":1}
I20260811 22:21:21.528846 30422 raft_consensus.cc:1261] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Refusing update from remote peer 783534c1e1894ac4925eecde99ff22b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:21.528867 30927 raft_consensus.cc:1261] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Refusing update from remote peer 783534c1e1894ac4925eecde99ff22b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:21.529235 31185 consensus_queue.cc:1035] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:21.529420 31209 consensus_queue.cc:1035] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:21.559062 30146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:21.559221 30146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:21.561333 30146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: Sending DeleteTablet for 3 replicas of tablet c48478a655d845ec8f1282d5f7c6bd08
I20260811 22:21:21.561959 30387 tablet_service.cc:1514] Processing DeleteTablet for tablet c48478a655d845ec8f1282d5f7c6bd08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:21 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:38298
I20260811 22:21:21.561959 30630 tablet_service.cc:1514] Processing DeleteTablet for tablet c48478a655d845ec8f1282d5f7c6bd08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:21 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:42564
I20260811 22:21:21.562064 30892 tablet_service.cc:1514] Processing DeleteTablet for tablet c48478a655d845ec8f1282d5f7c6bd08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:21 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:38276
I20260811 22:21:21.562179 31221 tablet_replica.cc:330] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: stopping tablet replica
I20260811 22:21:21.562186 31220 tablet_replica.cc:330] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: stopping tablet replica
I20260811 22:21:21.562307 31220 raft_consensus.cc:2229] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:21.562309 31221 raft_consensus.cc:2229] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:21.562348 31222 tablet_replica.cc:330] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: stopping tablet replica
I20260811 22:21:21.562438 31220 raft_consensus.cc:2258] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:21.562472 31222 raft_consensus.cc:2229] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:21.562516 31221 raft_consensus.cc:2258] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:21.562592 31222 raft_consensus.cc:2258] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:21.563218 31220 ts_tablet_manager.cc:1905] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:21.563292 31221 ts_tablet_manager.cc:1905] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:21.563309 31222 ts_tablet_manager.cc:1905] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:21.565732 31220 ts_tablet_manager.cc:1918] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:21.565815 31220 log.cc:1196] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/c48478a655d845ec8f1282d5f7c6bd08
I20260811 22:21:21.565857 31222 ts_tablet_manager.cc:1918] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:21.565857 31221 ts_tablet_manager.cc:1918] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:21.565943 31222 log.cc:1196] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/c48478a655d845ec8f1282d5f7c6bd08
I20260811 22:21:21.565944 31221 log.cc:1196] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/c48478a655d845ec8f1282d5f7c6bd08
I20260811 22:21:21.566128 31220 ts_tablet_manager.cc:1939] T c48478a655d845ec8f1282d5f7c6bd08 P 9236254bb50f4d599bfb9c43830f21c3: Deleting consensus metadata
I20260811 22:21:21.566200 31221 ts_tablet_manager.cc:1939] T c48478a655d845ec8f1282d5f7c6bd08 P 783534c1e1894ac4925eecde99ff22b5: Deleting consensus metadata
I20260811 22:21:21.566200 31222 ts_tablet_manager.cc:1939] T c48478a655d845ec8f1282d5f7c6bd08 P 593aa095192247e9816ed6f1ebaf230a: Deleting consensus metadata
I20260811 22:21:21.567008 30125 catalog_manager.cc:4841] TS 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385): tablet c48478a655d845ec8f1282d5f7c6bd08 (table test-table [id=23ad082b9bdc43f5a1910d3e5ae3e742]) successfully deleted
I20260811 22:21:21.567510 30125 catalog_manager.cc:4841] TS 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029): tablet c48478a655d845ec8f1282d5f7c6bd08 (table test-table [id=23ad082b9bdc43f5a1910d3e5ae3e742]) successfully deleted
I20260811 22:21:21.567544 30146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:21:21.567785 30127 catalog_manager.cc:4841] TS 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725): tablet c48478a655d845ec8f1282d5f7c6bd08 (table test-table [id=23ad082b9bdc43f5a1910d3e5ae3e742]) successfully deleted
W20260811 22:21:21.568059 30146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:21.572453 30387 tablet_service.cc:1467] Processing CreateTablet for tablet a1747c2074824061b9ee744b91f9760f (DEFAULT_TABLE table=test-table [id=7b79094874fa4290ad1f77c34ce83923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:21.572564 30630 tablet_service.cc:1467] Processing CreateTablet for tablet a1747c2074824061b9ee744b91f9760f (DEFAULT_TABLE table=test-table [id=7b79094874fa4290ad1f77c34ce83923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:21.572625 30892 tablet_service.cc:1467] Processing CreateTablet for tablet a1747c2074824061b9ee744b91f9760f (DEFAULT_TABLE table=test-table [id=7b79094874fa4290ad1f77c34ce83923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:21.572774 30387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1747c2074824061b9ee744b91f9760f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:21.572786 30630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1747c2074824061b9ee744b91f9760f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:21.572893 30892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1747c2074824061b9ee744b91f9760f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:21.574672 31240 tablet_bootstrap.cc:492] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: Bootstrap starting.
I20260811 22:21:21.575189 31241 tablet_bootstrap.cc:492] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3: Bootstrap starting.
I20260811 22:21:21.575282 31242 tablet_bootstrap.cc:492] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: Bootstrap starting.
I20260811 22:21:21.575647 31240 tablet_bootstrap.cc:654] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:21.576004 31242 tablet_bootstrap.cc:654] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:21.576004 31241 tablet_bootstrap.cc:654] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:21.576798 31240 tablet_bootstrap.cc:492] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: No bootstrap required, opened a new log
I20260811 22:21:21.576884 31240 ts_tablet_manager.cc:1397] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:21:21.577279 31242 tablet_bootstrap.cc:492] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: No bootstrap required, opened a new log
I20260811 22:21:21.577361 31242 ts_tablet_manager.cc:1397] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 22:21:21.577365 31241 tablet_bootstrap.cc:492] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3: No bootstrap required, opened a new log
I20260811 22:21:21.577346 31240 raft_consensus.cc:348] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.577442 31241 ts_tablet_manager.cc:1397] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:21:21.577471 31240 raft_consensus.cc:374] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:21.577512 31240 raft_consensus.cc:729] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593aa095192247e9816ed6f1ebaf230a, State: Initialized, Role: FOLLOWER
I20260811 22:21:21.577639 31240 consensus_queue.cc:260] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.577744 31242 raft_consensus.cc:348] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.577849 31242 raft_consensus.cc:374] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:21.577880 31242 raft_consensus.cc:729] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783534c1e1894ac4925eecde99ff22b5, State: Initialized, Role: FOLLOWER
I20260811 22:21:21.577853 31241 raft_consensus.cc:348] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.577915 31240 ts_tablet_manager.cc:1428] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:21.577953 31241 raft_consensus.cc:374] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:21.577982 31241 raft_consensus.cc:729] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9236254bb50f4d599bfb9c43830f21c3, State: Initialized, Role: FOLLOWER
I20260811 22:21:21.577991 31242 consensus_queue.cc:260] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.578099 31241 consensus_queue.cc:260] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.578234 31242 ts_tablet_manager.cc:1428] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 22:21:21.578343 31241 ts_tablet_manager.cc:1428] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:21:21.640707 31250 raft_consensus.cc:482] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:21.640887 31250 raft_consensus.cc:504] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.642350 31250 leader_election.cc:288] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385), 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725)
I20260811 22:21:21.647346 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.642978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:51261 (local address 127.26.125.2:46725)
0811 22:21:21.643646 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:21.643651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:21.643664 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:21:21.643720 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:21.643723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:21.643765 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:21.643823 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:21.644146 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.645259 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.646285 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.646696 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.646807 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.646896 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:21.647130 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 22:21:21.647211 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:21:21.647356 31272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.643214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:57631 (local address 127.26.125.1:38385)
0811 22:21:21.643519 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:21.643537 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:21.643553 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:21:21.643610 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:21.643613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:21.643663 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:21.643750 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:21.644264 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.645063 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.646250 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.646692 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.646808 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.646896 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:21.647138 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:21:21.647225 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
I20260811 22:21:21.647382 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.642703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46725 (local address 127.26.125.3:51261)
0811 22:21:21.643079 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:21.643094 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:21.643201 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:21.643886 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:21.643889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:21.643907 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:21.644032 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:21.644037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.645381 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.645385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:21.646163 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:21.646172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.646808 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.646811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:21.647012 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.647032 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:21:21.647099 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:21:21.647182 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":64,"thread_start_us":129,"threads_started":1}
I20260811 22:21:21.647653 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.642540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:38385 (local address 127.26.125.3:57631)
0811 22:21:21.643011 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:21.643026 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:21.643146 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:21.643801 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:21.643808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:21.643831 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:21.644108 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:21.644116 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.645183 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.645186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:21.646121 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:21.646132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.646809 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.646813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:21.647015 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.647042 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:21:21.647106 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 22:21:21.647184 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":90,"thread_start_us":79,"threads_started":1}
I20260811 22:21:21.648213 30420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1747c2074824061b9ee744b91f9760f" candidate_uuid: "593aa095192247e9816ed6f1ebaf230a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9236254bb50f4d599bfb9c43830f21c3" is_pre_election: true
I20260811 22:21:21.648280 30661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1747c2074824061b9ee744b91f9760f" candidate_uuid: "593aa095192247e9816ed6f1ebaf230a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783534c1e1894ac4925eecde99ff22b5" is_pre_election: true
I20260811 22:21:21.648376 30420 raft_consensus.cc:2454] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 593aa095192247e9816ed6f1ebaf230a in term 0.
I20260811 22:21:21.648427 30661 raft_consensus.cc:2454] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 593aa095192247e9816ed6f1ebaf230a in term 0.
I20260811 22:21:21.648759 30800 leader_election.cc:302] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 593aa095192247e9816ed6f1ebaf230a, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:21.649042 31250 raft_consensus.cc:2781] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:21.649101 31250 raft_consensus.cc:482] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:21.649128 31250 raft_consensus.cc:3037] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:21.650175 31250 raft_consensus.cc:504] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.650553 31250 leader_election.cc:288] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [CANDIDATE]: Term 1 election: Requested vote from peers 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385), 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725)
I20260811 22:21:21.650938 30420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1747c2074824061b9ee744b91f9760f" candidate_uuid: "593aa095192247e9816ed6f1ebaf230a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9236254bb50f4d599bfb9c43830f21c3"
I20260811 22:21:21.650974 30661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1747c2074824061b9ee744b91f9760f" candidate_uuid: "593aa095192247e9816ed6f1ebaf230a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783534c1e1894ac4925eecde99ff22b5"
I20260811 22:21:21.651065 30420 raft_consensus.cc:3037] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:21.651089 30661 raft_consensus.cc:3037] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:21.652031 30661 raft_consensus.cc:2454] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 593aa095192247e9816ed6f1ebaf230a in term 1.
I20260811 22:21:21.652083 30420 raft_consensus.cc:2454] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 593aa095192247e9816ed6f1ebaf230a in term 1.
I20260811 22:21:21.652360 30803 leader_election.cc:302] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 593aa095192247e9816ed6f1ebaf230a, 783534c1e1894ac4925eecde99ff22b5, 9236254bb50f4d599bfb9c43830f21c3; no voters:
I20260811 22:21:21.652559 31250 raft_consensus.cc:2781] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:21.652810 31250 raft_consensus.cc:686] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 1 LEADER]: Becoming Leader. State: Replica: 593aa095192247e9816ed6f1ebaf230a, State: Running, Role: LEADER
I20260811 22:21:21.653087 31250 consensus_queue.cc:237] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } }
I20260811 22:21:21.655838 30144 catalog_manager.cc:5495] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a reported cstate change: term changed from 0 to 1, leader changed from <none> to 593aa095192247e9816ed6f1ebaf230a (127.26.125.3). New cstate: current_term: 1 leader_uuid: "593aa095192247e9816ed6f1ebaf230a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "593aa095192247e9816ed6f1ebaf230a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 46029 } health_report { overall_health: HEALTHY } } }
I20260811 22:21:21.689977 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.686326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.26.125.3:46029)
0811 22:21:21.686641 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:21.686647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:21.686660 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:21:21.686737 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:21.686739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:21.686792 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:21.686926 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:21.687147 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.687726 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.688484 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.688733 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.688804 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.689002 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:21.689595 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:21.689688 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:21.689755 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 22:21:21.689806 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":131,"threads_started":1}
I20260811 22:21:21.700595 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.697331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:52889 (local address 127.26.125.2:46725)
0811 22:21:21.697481 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:21.697486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:21.697699 (+ 213us) server_negotiation.cc:407] Connection header received
0811 22:21:21.697833 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:21.697836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:21.697889 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:21.697968 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:21.698228 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.698757 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.699656 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.699879 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.699961 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.700230 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:21.700301 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:21.700348 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:21.700407 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:21:21.700455 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:21:21.700618 31284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.697257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46725 (local address 127.26.125.3:52889)
0811 22:21:21.697616 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:21.697631 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:21.697745 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:21.698011 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:21.698014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:21.698023 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:21.698140 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:21.698145 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.698841 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.698843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:21.699523 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:21.699533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.700008 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.700011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:21.700077 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.700127 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:21.700355 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:21.700358 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:21:21.700399 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 22:21:21.700462 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":133,"threads_started":1}
I20260811 22:21:21.707013 30420 raft_consensus.cc:1261] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3 [term 1 FOLLOWER]: Refusing update from remote peer 593aa095192247e9816ed6f1ebaf230a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:21.707013 30661 raft_consensus.cc:1261] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Refusing update from remote peer 593aa095192247e9816ed6f1ebaf230a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:21.707525 31250 consensus_queue.cc:1035] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [LEADER]: Connected to new peer: Peer: permanent_uuid: "783534c1e1894ac4925eecde99ff22b5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:21.707748 31275 consensus_queue.cc:1035] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9236254bb50f4d599bfb9c43830f21c3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 38385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:21.737766 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.734110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:55121 (local address 127.26.125.3:46029)
0811 22:21:21.734246 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:21.734251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:21.734512 (+ 261us) server_negotiation.cc:407] Connection header received
0811 22:21:21.734647 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:21.734650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:21.734701 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:21.734770 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:21.735235 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.736059 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.736885 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.737107 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.737186 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.737394 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:21.737464 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:21.737520 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:21.737612 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 22:21:21.737659 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:21:21.737797 31291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:21.734008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:46029 (local address 127.26.125.2:55121)
0811 22:21:21.734409 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:21.734425 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:21.734548 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:21.734837 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:21.734841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:21.734869 (+ 28us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:21.734993 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:21.735000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.736163 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.736165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:21.736785 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:21.736792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:21.737197 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:21.737200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:21.737262 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:21.737293 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:21.737550 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:21.737555 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:21:21.737603 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 22:21:21.737676 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":140,"threads_started":1}
W20260811 22:21:21.738669 31189 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:21:23.295683 30144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:23.295840 30144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:23.298826 30144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c893d7be21414c148f4e70188a652c04: Sending DeleteTablet for 3 replicas of tablet a1747c2074824061b9ee744b91f9760f
I20260811 22:21:23.299471 30387 tablet_service.cc:1514] Processing DeleteTablet for tablet a1747c2074824061b9ee744b91f9760f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:23 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:38298
I20260811 22:21:23.299492 30630 tablet_service.cc:1514] Processing DeleteTablet for tablet a1747c2074824061b9ee744b91f9760f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:23 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:42564
I20260811 22:21:23.299526 30892 tablet_service.cc:1514] Processing DeleteTablet for tablet a1747c2074824061b9ee744b91f9760f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:23 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:38276
I20260811 22:21:23.299850 31338 tablet_replica.cc:330] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: stopping tablet replica
I20260811 22:21:23.299911 31334 tablet_replica.cc:330] T a1747c2074824061b9ee744b91f9760f P 9236254bb50f4d599bfb9c43830f21c3: stopping tablet replica
I20260811 22:21:23.300004 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30279
I20260811 22:21:23.300029 31338 raft_consensus.cc:2229] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:23.312275 31338 raft_consensus.cc:2258] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:23.312642 31339 tablet_replica.cc:330] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: stopping tablet replica
I20260811 22:21:23.312898 31339 raft_consensus.cc:2229] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:23.313161 31338 ts_tablet_manager.cc:1905] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:23.313405 31339 raft_consensus.cc:2258] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:23.314491 31339 ts_tablet_manager.cc:1905] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:23.316123 31338 ts_tablet_manager.cc:1918] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:21:23.316226 31338 log.cc:1196] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/a1747c2074824061b9ee744b91f9760f
I20260811 22:21:23.316550 31338 ts_tablet_manager.cc:1939] T a1747c2074824061b9ee744b91f9760f P 783534c1e1894ac4925eecde99ff22b5: Deleting consensus metadata
I20260811 22:21:23.317363 31339 ts_tablet_manager.cc:1918] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:21:23.317492 31339 log.cc:1196] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/a1747c2074824061b9ee744b91f9760f
I20260811 22:21:23.317512 30127 catalog_manager.cc:4841] TS 783534c1e1894ac4925eecde99ff22b5 (127.26.125.2:46725): tablet a1747c2074824061b9ee744b91f9760f (table test-table [id=7b79094874fa4290ad1f77c34ce83923]) successfully deleted
I20260811 22:21:23.317796 31339 ts_tablet_manager.cc:1939] T a1747c2074824061b9ee744b91f9760f P 593aa095192247e9816ed6f1ebaf230a: Deleting consensus metadata
I20260811 22:21:23.318504 30125 catalog_manager.cc:4841] TS 593aa095192247e9816ed6f1ebaf230a (127.26.125.3:46029): tablet a1747c2074824061b9ee744b91f9760f (table test-table [id=7b79094874fa4290ad1f77c34ce83923]) successfully deleted
W20260811 22:21:23.323885 30125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:38385 (error 108)
I20260811 22:21:23.324131 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30524
I20260811 22:21:23.344252 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:23.331110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:38385 (local address 127.0.0.1:38320)
0811 22:21:23.343969 (+ 12859us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:23.344115 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12737,"thread_start_us":1051,"threads_started":1}
W20260811 22:21:23.344501 30125 catalog_manager.cc:4584] TS 9236254bb50f4d599bfb9c43830f21c3 (127.26.125.1:38385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1747c2074824061b9ee744b91f9760f: Network error: Client connection negotiation failed: client connection to 127.26.125.1:38385: connect: Connection refused (error 111)
I20260811 22:21:23.351348 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30772
I20260811 22:21:23.368652 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30099
2026-08-11T22:21:23Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6970 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:23 dist-test-slave-5bn2 krb5kdc[31349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:23 dist-test-slave-5bn2 krb5kdc[31349](info): set up 2 sockets
Aug 11 22:21:23 dist-test-slave-5bn2 krb5kdc[31349](info): commencing operation
krb5kdc: starting...
W20260811 22:21:24.571959 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:24 dist-test-slave-5bn2 krb5kdc[31349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:24.701661 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41687
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:41687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:24.817974 31369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:24.818280 31369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:24.818341 31369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:24.822412 31369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:24.822496 31369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:24.822525 31369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:24.822546 31369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:24.822570 31369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:24.828065 31369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:41687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:24.829206 31369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:24.830308 31369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:24.838976 31369 server_base.cc:1034] running on GCE node
W20260811 22:21:24.839010 31378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:24.839018 31377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:24.838989 31381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:24.839913 31369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:24.841118 31369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:24.842275 31369 hybrid_clock.cc:648] HybridClock initialized: now 1786486884842247 us; error 44 us; skew 500 ppm
Aug 11 22:21:24 dist-test-slave-5bn2 krb5kdc[31349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:24.849180 31369 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:24.850566 31369 webserver.cc:460] Webserver started at http://127.26.125.62:41287/ using document root <none> and password file <none>
I20260811 22:21:24.851249 31369 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:24.851332 31369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:24.851563 31369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:24.853816 31369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "ed094f0e18ee4571a29fb1ca22028b01"
format_stamp: "Formatted at 2026-08-11 22:21:24 on dist-test-slave-5bn2"
server_key: "5c4b050aaa1f06c475f64475fdee3d73"
server_key_iv: "77c2b14a8946877162944583dcae732d"
server_key_version: "encryptionkey@0"
I20260811 22:21:24.854429 31369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "ed094f0e18ee4571a29fb1ca22028b01"
format_stamp: "Formatted at 2026-08-11 22:21:24 on dist-test-slave-5bn2"
server_key: "5c4b050aaa1f06c475f64475fdee3d73"
server_key_iv: "77c2b14a8946877162944583dcae732d"
server_key_version: "encryptionkey@0"
I20260811 22:21:24.858451 31369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 22:21:24.861043 31396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:24.862339 31369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:24.862465 31369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "ed094f0e18ee4571a29fb1ca22028b01"
format_stamp: "Formatted at 2026-08-11 22:21:24 on dist-test-slave-5bn2"
server_key: "5c4b050aaa1f06c475f64475fdee3d73"
server_key_iv: "77c2b14a8946877162944583dcae732d"
server_key_version: "encryptionkey@0"
I20260811 22:21:24.862573 31369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:24.893950 31369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:24.897661 31369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:24.897876 31369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:24.905406 31369 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:41687
I20260811 22:21:24.905498 31493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:41687 every 8 connection(s)
I20260811 22:21:24.906515 31369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:24.909391 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31369
I20260811 22:21:24.909502 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:24.909704 31494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:24.909829 27124 external_mini_cluster.cc:1428] Setting key 76612f2080352cee5fdc6e5fd7c41759
I20260811 22:21:24.917686 31494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01: Bootstrap starting.
Aug 11 22:21:24 dist-test-slave-5bn2 krb5kdc[31349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:24.920163 31494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:24.921103 31494 log.cc:824] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:24.923518 31494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01: No bootstrap required, opened a new log
I20260811 22:21:24.925132 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:24.911463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.26.125.62:41687)
0811 22:21:24.911954 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:24.911975 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:24.912023 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:21:24.912828 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:24.912862 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:24.913335 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:24.913743 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:24.914453 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:24.915979 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:24.916864 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:24.917549 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:24.917709 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:24.920168 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:24.920175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:24.920194 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:24.920231 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:24.921979 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:24.922401 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:24.922407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:24.922413 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:24.922557 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:24.922794 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:24.922797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:24.922799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:24.923352 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:24.923543 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:24.923834 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:21:24.924163 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":128,"threads_started":1}
I20260811 22:21:24.927835 31494 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed094f0e18ee4571a29fb1ca22028b01" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41687 } }
I20260811 22:21:24.928059 31494 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:24.928100 31494 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed094f0e18ee4571a29fb1ca22028b01, State: Initialized, Role: FOLLOWER
I20260811 22:21:24.928553 31494 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed094f0e18ee4571a29fb1ca22028b01" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41687 } }
I20260811 22:21:24.928681 31494 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:24.928728 31494 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:24.928798 31494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:24.929667 31494 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed094f0e18ee4571a29fb1ca22028b01" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41687 } }
I20260811 22:21:24.929987 31494 leader_election.cc:302] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed094f0e18ee4571a29fb1ca22028b01; no voters:
I20260811 22:21:24.930315 31494 leader_election.cc:288] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:24.930430 31501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:24.930691 31501 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [term 1 LEADER]: Becoming Leader. State: Replica: ed094f0e18ee4571a29fb1ca22028b01, State: Running, Role: LEADER
I20260811 22:21:24.931304 31501 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed094f0e18ee4571a29fb1ca22028b01" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41687 } }
I20260811 22:21:24.931603 31494 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:24.932981 31503 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed094f0e18ee4571a29fb1ca22028b01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed094f0e18ee4571a29fb1ca22028b01" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41687 } } }
I20260811 22:21:24.933007 31502 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed094f0e18ee4571a29fb1ca22028b01. Latest consensus state: current_term: 1 leader_uuid: "ed094f0e18ee4571a29fb1ca22028b01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed094f0e18ee4571a29fb1ca22028b01" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41687 } } }
I20260811 22:21:24.933122 31502 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:24.933122 31503 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:24.933545 31523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:24.936476 31523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:24.942315 31523 catalog_manager.cc:1347] Generated new cluster ID: 97bcc973595f46d78de1923af8cca239
I20260811 22:21:24.942399 31523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:24.956759 31523 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:24.958134 31523 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:24.963213 31523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed094f0e18ee4571a29fb1ca22028b01: Generated new TSK 0
I20260811 22:21:24.963968 31523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:24.999259 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41687
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:25.117627 31532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:25.117918 31532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:25.117959 31532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:25.122063 31532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:25.122139 31532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:25.122198 31532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:25.127648 31532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:25.128736 31532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:25.129804 31532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:25.140046 31532 server_base.cc:1034] running on GCE node
W20260811 22:21:25.140002 31540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:25.139983 31541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:25.139986 31544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:25.140604 31532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:25.141252 31532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:25.142413 31532 hybrid_clock.cc:648] HybridClock initialized: now 1786486885142390 us; error 37 us; skew 500 ppm
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[31349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486885, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:25.145254 31532 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:25.146207 31532 webserver.cc:460] Webserver started at http://127.26.125.1:42169/ using document root <none> and password file <none>
I20260811 22:21:25.146759 31532 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:25.146817 31532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:25.147053 31532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:25.149117 31532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "ef39dfd5e1df41e79460c34e3ada6860"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "dd82fe82d97e5a32cd3b489a4d3936f2"
server_key_iv: "6b9447847e0075caec8772509f585041"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.149672 31532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "ef39dfd5e1df41e79460c34e3ada6860"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "dd82fe82d97e5a32cd3b489a4d3936f2"
server_key_iv: "6b9447847e0075caec8772509f585041"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.153587 31532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:25.156147 31557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.157414 31532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:21:25.157519 31532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "ef39dfd5e1df41e79460c34e3ada6860"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "dd82fe82d97e5a32cd3b489a4d3936f2"
server_key_iv: "6b9447847e0075caec8772509f585041"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.157631 31532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:25.172138 31532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:25.175542 31532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:25.175724 31532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:25.176344 31532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:25.177246 31532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:25.177294 31532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.177335 31532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:25.177349 31532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.185894 31532 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:35655
I20260811 22:21:25.185917 31743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:35655 every 8 connection(s)
I20260811 22:21:25.186923 31532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[31349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486885, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:25.196942 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31532
I20260811 22:21:25.197047 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:25.197350 27124 external_mini_cluster.cc:1428] Setting key f7a8d4a8f3547018e71162b067131cd8
I20260811 22:21:25.200631 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:25.188856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:39791 (local address 127.26.125.62:41687)
0811 22:21:25.188990 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:25.188995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:25.190098 (+ 1103us) server_negotiation.cc:407] Connection header received
0811 22:21:25.190994 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:25.190997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:25.191054 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:25.191126 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:25.192328 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.192894 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.194040 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.194273 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.194366 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:25.196852 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:25.196856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:25.196858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:25.196883 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:25.198400 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:25.199006 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:25.199009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:25.199010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:25.199099 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:25.199492 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:25.199495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:25.199496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:25.199683 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:25.199757 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:25.200382 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 22:21:25.200504 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:21:25.201643 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:25.189275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41687 (local address 127.26.125.1:39791)
0811 22:21:25.189928 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:25.189970 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:25.190769 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:25.191315 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:25.191323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:25.191681 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:25.192175 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:25.192188 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.193027 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.193029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:25.193916 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:25.193925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.194420 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.194424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:25.194591 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:25.195202 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:25.195219 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:25.196682 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:25.198518 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:25.198532 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:25.198552 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:25.198896 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:25.199188 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:25.199191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:25.199193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:25.199404 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:25.199777 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:25.199784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:25.200130 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 22:21:25.200338 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:21:25.200714 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":111,"threads_started":1}
I20260811 22:21:25.202883 31745 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41687
I20260811 22:21:25.203151 31745 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:25.203761 31745 heartbeater.cc:511] Master 127.26.125.62:41687 requested a full tablet report, sending...
I20260811 22:21:25.205471 31423 ts_manager.cc:194] Registered new tserver with Master: ef39dfd5e1df41e79460c34e3ada6860 (127.26.125.1:35655)
I20260811 22:21:25.206664 31423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:39791
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:25.225481 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41687
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:25.341744 31752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:25.342059 31752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:25.342103 31752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:25.346305 31752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:25.346405 31752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:25.346483 31752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:25.352129 31752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:25.353308 31752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:25.354379 31752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:25.364642 31765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:25.364732 31752 server_base.cc:1034] running on GCE node
W20260811 22:21:25.364642 31758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:25.364660 31759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:25.365393 31752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:25.366124 31752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:25.367286 31752 hybrid_clock.cc:648] HybridClock initialized: now 1786486885367262 us; error 35 us; skew 500 ppm
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[31349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486885, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:25.370179 31752 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:25.371199 31752 webserver.cc:460] Webserver started at http://127.26.125.2:38749/ using document root <none> and password file <none>
I20260811 22:21:25.371783 31752 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:25.371853 31752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:25.372082 31752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:25.374226 31752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "9fc9326dd2f64af589a04147a08d7467"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "cc0a13fed2d1a62616bdc1e6aaeb96a5"
server_key_iv: "15baa72a34aa80279676fe73a3fd09f0"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.374799 31752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "9fc9326dd2f64af589a04147a08d7467"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "cc0a13fed2d1a62616bdc1e6aaeb96a5"
server_key_iv: "15baa72a34aa80279676fe73a3fd09f0"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.378659 31752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:21:25.381230 31773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.382566 31752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:25.382704 31752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "9fc9326dd2f64af589a04147a08d7467"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "cc0a13fed2d1a62616bdc1e6aaeb96a5"
server_key_iv: "15baa72a34aa80279676fe73a3fd09f0"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.382818 31752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:25.397045 31752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:25.400374 31752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:25.400556 31752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:25.401170 31752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:25.402062 31752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:25.402134 31752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.402200 31752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:25.402225 31752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.411042 31752 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:42183
I20260811 22:21:25.411078 31979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:42183 every 8 connection(s)
I20260811 22:21:25.411999 31752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:25.413072 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31752
I20260811 22:21:25.413192 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:25.413486 27124 external_mini_cluster.cc:1428] Setting key e62039d4f8fb8c0c3c97ebcc80c1bc8f
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[31349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486885, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:25.425864 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:25.414094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:55579 (local address 127.26.125.62:41687)
0811 22:21:25.414240 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:25.414245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:25.415472 (+ 1227us) server_negotiation.cc:407] Connection header received
0811 22:21:25.416407 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:25.416411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:25.416467 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:25.416545 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:25.417754 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.418286 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.419506 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.419737 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.419827 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:25.422485 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:25.422489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:25.422491 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:25.422517 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:25.423911 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:25.424450 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:25.424453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:25.424454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:25.424538 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:25.424820 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:25.424823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:25.424824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:25.425017 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:25.425091 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:25.425565 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 22:21:25.425686 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:21:25.426792 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:25.414531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41687 (local address 127.26.125.2:55579)
0811 22:21:25.415326 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:25.415382 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:25.416203 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:25.416724 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:25.416732 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:25.417108 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:25.417607 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:25.417620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.418409 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.418412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:25.419379 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:25.419388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.419882 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.419886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:25.420063 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:25.420663 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:25.420680 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:25.422311 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:25.424032 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:25.424038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:25.424053 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:25.424334 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:25.424612 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:25.424614 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:25.424615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:25.424754 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:25.425089 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:25.425094 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:25.425359 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 22:21:25.425517 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 22:21:25.425798 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":125,"threads_started":1}
I20260811 22:21:25.427978 31980 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41687
I20260811 22:21:25.428247 31980 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:25.428761 31980 heartbeater.cc:511] Master 127.26.125.62:41687 requested a full tablet report, sending...
I20260811 22:21:25.429854 31423 ts_manager.cc:194] Registered new tserver with Master: 9fc9326dd2f64af589a04147a08d7467 (127.26.125.2:42183)
I20260811 22:21:25.430429 31423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:55579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:25.442392 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41687
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:25.561672 31985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:25.561942 31985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:25.561993 31985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:25.566195 31985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:25.566264 31985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:25.566329 31985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:25.571954 31985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:25.573164 31985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:25.574271 31985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:25.584506 32010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:25.584556 32008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:25.584556 32001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:25.584759 31985 server_base.cc:1034] running on GCE node
I20260811 22:21:25.585273 31985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:25.586019 31985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:25.587183 31985 hybrid_clock.cc:648] HybridClock initialized: now 1786486885587147 us; error 44 us; skew 500 ppm
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[31349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486885, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:25.590219 31985 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:25.591401 31985 webserver.cc:460] Webserver started at http://127.26.125.3:41263/ using document root <none> and password file <none>
I20260811 22:21:25.592100 31985 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:25.592170 31985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:25.592386 31985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:25.594921 31985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "af36abf566224c62b77e3f6c4d93a806"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "96502e2ab798394675c2a15dafba5a85"
server_key_iv: "e904de7409310fcc5358f98362de3a99"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.595548 31985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "af36abf566224c62b77e3f6c4d93a806"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "96502e2ab798394675c2a15dafba5a85"
server_key_iv: "e904de7409310fcc5358f98362de3a99"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.599555 31985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:25.602265 32022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.603770 31985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:25.603929 31985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "af36abf566224c62b77e3f6c4d93a806"
format_stamp: "Formatted at 2026-08-11 22:21:25 on dist-test-slave-5bn2"
server_key: "96502e2ab798394675c2a15dafba5a85"
server_key_iv: "e904de7409310fcc5358f98362de3a99"
server_key_version: "encryptionkey@0"
I20260811 22:21:25.604171 31985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:25.627079 31985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:25.630103 31985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:25.630291 31985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:25.630949 31985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:25.631843 31985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:25.631893 31985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.631949 31985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:25.631978 31985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:25.640802 31985 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:41587
I20260811 22:21:25.640830 32214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:41587 every 8 connection(s)
I20260811 22:21:25.641743 31985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:25.650236 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31985
I20260811 22:21:25.650367 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:25.650664 27124 external_mini_cluster.cc:1428] Setting key bc7a04009db2136c5fe88b77859070af
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[31349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486885, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:25.657002 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:25.643683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:35529 (local address 127.26.125.62:41687)
0811 22:21:25.643829 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:25.643834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:25.645026 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 22:21:25.645849 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:25.645852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:25.645901 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:25.645977 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:25.647243 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.647787 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.648953 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.649195 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.649284 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:25.652755 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:25.652759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:25.652761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:25.652794 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:25.654705 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:25.655366 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:25.655369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:25.655371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:25.655453 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:25.655833 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:25.655836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:25.655837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:25.656047 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:25.656134 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:25.656703 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 22:21:25.656839 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:21:25.657874 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:25.644123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41687 (local address 127.26.125.3:35529)
0811 22:21:25.644849 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:25.644898 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:25.645666 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:25.646141 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:25.646148 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:25.646520 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:25.647088 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:25.647104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.647888 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.647890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:25.648826 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:25.648836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:25.649325 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:25.649327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:25.649473 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:25.650134 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:25.650155 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:25.652535 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:25.654831 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:25.654838 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:25.654886 (+ 48us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:25.655267 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:25.655570 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:25.655573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:25.655575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:25.655731 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:25.656153 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:25.656160 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:25.656438 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 22:21:25.656625 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 22:21:25.656987 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":112,"threads_started":1}
I20260811 22:21:25.659088 32216 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41687
I20260811 22:21:25.659356 32216 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:25.659941 32216 heartbeater.cc:511] Master 127.26.125.62:41687 requested a full tablet report, sending...
I20260811 22:21:25.661372 31423 ts_manager.cc:194] Registered new tserver with Master: af36abf566224c62b77e3f6c4d93a806 (127.26.125.3:41587)
I20260811 22:21:25.661971 31423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:35529
I20260811 22:21:25.665472 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:21:25.682912 31498 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:25.681360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.26.125.62:41687)
0811 22:21:25.681572 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:25.681577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:25.681673 (+ 96us) server_negotiation.cc:407] Connection header received
0811 22:21:25.681823 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:25.681827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:25.681873 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:25.681942 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:25.682748 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60150: BlockingRecv error: recv got EOF from 127.0.0.1:60150 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:21:25.684142 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31532
I20260811 22:21:25.699927 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31752
I20260811 22:21:25.718014 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31985
I20260811 22:21:25.732364 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31369
2026-08-11T22:21:25Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2353 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[32234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[32234](info): set up 2 sockets
Aug 11 22:21:25 dist-test-slave-5bn2 krb5kdc[32234](info): commencing operation
krb5kdc: starting...
W20260811 22:21:26.928357 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:27 dist-test-slave-5bn2 krb5kdc[32234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:27.062134 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40625
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:27.180174 32256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:27.180477 32256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:27.180523 32256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:27.184725 32256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:27.184804 32256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:27.184823 32256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:27.184839 32256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:27.184854 32256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:27.190515 32256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:27.191780 32256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:27.192909 32256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:27.201923 32262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:27.201922 32263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:27.202067 32256 server_base.cc:1034] running on GCE node
W20260811 22:21:27.201918 32267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:27.203044 32256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:27.204237 32256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:27.205420 32256 hybrid_clock.cc:648] HybridClock initialized: now 1786486887205398 us; error 32 us; skew 500 ppm
Aug 11 22:21:27 dist-test-slave-5bn2 krb5kdc[32234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486887, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:27.208365 32256 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:27.209342 32256 webserver.cc:460] Webserver started at http://127.26.125.62:38905/ using document root <none> and password file <none>
I20260811 22:21:27.209918 32256 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:27.209990 32256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:27.210233 32256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:27.212461 32256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "a1111d555d9a42d592d5a03b804fce1f"
format_stamp: "Formatted at 2026-08-11 22:21:27 on dist-test-slave-5bn2"
server_key: "ed6936beedb316db99e50c5e2d2590f4"
server_key_iv: "c7e2d239bdb80c41074d7baab2ad3d70"
server_key_version: "encryptionkey@0"
I20260811 22:21:27.213271 32256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "a1111d555d9a42d592d5a03b804fce1f"
format_stamp: "Formatted at 2026-08-11 22:21:27 on dist-test-slave-5bn2"
server_key: "ed6936beedb316db99e50c5e2d2590f4"
server_key_iv: "c7e2d239bdb80c41074d7baab2ad3d70"
server_key_version: "encryptionkey@0"
I20260811 22:21:27.217602 32256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:27.220387 32281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:27.221711 32256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:27.221843 32256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "a1111d555d9a42d592d5a03b804fce1f"
format_stamp: "Formatted at 2026-08-11 22:21:27 on dist-test-slave-5bn2"
server_key: "ed6936beedb316db99e50c5e2d2590f4"
server_key_iv: "c7e2d239bdb80c41074d7baab2ad3d70"
server_key_version: "encryptionkey@0"
I20260811 22:21:27.221964 32256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:27.250738 32256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:27.254628 32256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:27.254906 32256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:27.262673 32256 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:40625
I20260811 22:21:27.263010 32383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:40625 every 8 connection(s)
I20260811 22:21:27.263849 32256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:27.267457 32386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:27.270749 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 32256
I20260811 22:21:27.270951 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:27.271241 27124 external_mini_cluster.cc:1428] Setting key c7431c94c7993cf1b3cf2674070fbade
I20260811 22:21:27.274469 32386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f: Bootstrap starting.
I20260811 22:21:27.277354 32386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:27.278383 32386 log.cc:824] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:27.280719 32386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f: No bootstrap required, opened a new log
Aug 11 22:21:27 dist-test-slave-5bn2 krb5kdc[32234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:27.284385 32386 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1111d555d9a42d592d5a03b804fce1f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40625 } }
I20260811 22:21:27.284667 32386 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:27.284734 32386 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1111d555d9a42d592d5a03b804fce1f, State: Initialized, Role: FOLLOWER
I20260811 22:21:27.285279 32386 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1111d555d9a42d592d5a03b804fce1f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40625 } }
I20260811 22:21:27.285468 32386 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:27.285543 32386 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:27.285648 32386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:27.286935 32386 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1111d555d9a42d592d5a03b804fce1f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40625 } }
I20260811 22:21:27.287374 32386 leader_election.cc:302] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1111d555d9a42d592d5a03b804fce1f; no voters:
I20260811 22:21:27.287789 32386 leader_election.cc:288] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:27.287905 32399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:27.288081 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:27.272723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.26.125.62:40625)
0811 22:21:27.273253 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:27.273270 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:27.273313 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:21:27.274090 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:27.274115 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:27.274769 (+ 654us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:27.275178 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:27.276170 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:27.277253 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:27.278520 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:27.279121 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:27.279295 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:27.281900 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:27.281906 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:27.281924 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:27.281958 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:27.284286 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:27.284800 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:27.284808 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:27.284815 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:27.285001 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:27.285314 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:27.285317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:27.285319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:27.285949 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:27.286166 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:27.286477 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:21:27.286902 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":149,"threads_started":1}
I20260811 22:21:27.288187 32399 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [term 1 LEADER]: Becoming Leader. State: Replica: a1111d555d9a42d592d5a03b804fce1f, State: Running, Role: LEADER
I20260811 22:21:27.288601 32399 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1111d555d9a42d592d5a03b804fce1f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40625 } }
I20260811 22:21:27.289362 32386 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:27.291025 32401 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1111d555d9a42d592d5a03b804fce1f. Latest consensus state: current_term: 1 leader_uuid: "a1111d555d9a42d592d5a03b804fce1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1111d555d9a42d592d5a03b804fce1f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40625 } } }
I20260811 22:21:27.291177 32401 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:27.291996 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:21:27.292009 32400 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1111d555d9a42d592d5a03b804fce1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1111d555d9a42d592d5a03b804fce1f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40625 } } }
I20260811 22:21:27.292208 32400 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:27.292284 32416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:27.295814 32416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:27.302034 32416 catalog_manager.cc:1347] Generated new cluster ID: 3f6551e3bf644a9f8a2dad9bb70acbe9
I20260811 22:21:27.302131 32416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:27.311936 32416 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:27.313338 32416 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:27.315089 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:27.309672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.26.125.62:40625)
0811 22:21:27.309794 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:27.309799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:27.310211 (+ 412us) server_negotiation.cc:407] Connection header received
0811 22:21:27.310289 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:27.310293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:27.310417 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:27.310529 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:27.311229 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:27.311888 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:27.312788 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:27.313047 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:27.313139 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:27.314066 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:27.314069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:27.314071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:27.314570 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:27.314604 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:27.314622 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:27.314705 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:27.314817 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 22:21:27.314935 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:21:27.325103 32416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1111d555d9a42d592d5a03b804fce1f: Generated new TSK 0
I20260811 22:21:27.326009 32416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:21:27.337154 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 32256
2026-08-11T22:21:27Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1602 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:27 dist-test-slave-5bn2 krb5kdc[32447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:27 dist-test-slave-5bn2 krb5kdc[32447](info): set up 2 sockets
Aug 11 22:21:27 dist-test-slave-5bn2 krb5kdc[32447](info): commencing operation
krb5kdc: starting...
W20260811 22:21:28.549055 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:28 dist-test-slave-5bn2 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:28.684413 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41427
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:41427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:28.803833 32472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:28.804158 32472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:28.804201 32472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:28.808293 32472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:28.808374 32472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:28.808393 32472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:28.808408 32472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:28.808422 32472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:28.814285 32472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:41427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:28.815580 32472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:28.816622 32472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:28.825546 32487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:28.825546 32491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:28.825659 32472 server_base.cc:1034] running on GCE node
W20260811 22:21:28.825548 32486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:28.826546 32472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:28.827862 32472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:28.829066 32472 hybrid_clock.cc:648] HybridClock initialized: now 1786486888829020 us; error 37 us; skew 500 ppm
Aug 11 22:21:28 dist-test-slave-5bn2 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486888, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:28.832163 32472 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:28.833138 32472 webserver.cc:460] Webserver started at http://127.26.125.62:34819/ using document root <none> and password file <none>
I20260811 22:21:28.833736 32472 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:28.833810 32472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:28.834049 32472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:28.836313 32472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "cf3050063ee446b4af02d8ea559720b1"
format_stamp: "Formatted at 2026-08-11 22:21:28 on dist-test-slave-5bn2"
server_key: "5aff2aa8d3802c0e9beaa197ec81e980"
server_key_iv: "f502374dd24cdc7e43e4cef54a3cf8d3"
server_key_version: "encryptionkey@0"
I20260811 22:21:28.836930 32472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "cf3050063ee446b4af02d8ea559720b1"
format_stamp: "Formatted at 2026-08-11 22:21:28 on dist-test-slave-5bn2"
server_key: "5aff2aa8d3802c0e9beaa197ec81e980"
server_key_iv: "f502374dd24cdc7e43e4cef54a3cf8d3"
server_key_version: "encryptionkey@0"
I20260811 22:21:28.841122 32472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:28.843884 32502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:28.845274 32472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:21:28.845405 32472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "cf3050063ee446b4af02d8ea559720b1"
format_stamp: "Formatted at 2026-08-11 22:21:28 on dist-test-slave-5bn2"
server_key: "5aff2aa8d3802c0e9beaa197ec81e980"
server_key_iv: "f502374dd24cdc7e43e4cef54a3cf8d3"
server_key_version: "encryptionkey@0"
I20260811 22:21:28.845516 32472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:28.858265 32472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:28.861524 32472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:28.861733 32472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:28.868875 32472 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:41427
I20260811 22:21:28.869151 32604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:41427 every 8 connection(s)
I20260811 22:21:28.869905 32472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:28.872444 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 32472
I20260811 22:21:28.872578 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:28.872910 27124 external_mini_cluster.cc:1428] Setting key 70d50082f9aa0624b1c08bbdc6abc3aa
I20260811 22:21:28.873221 32605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:28.880018 32605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1: Bootstrap starting.
Aug 11 22:21:28 dist-test-slave-5bn2 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:28.882555 32605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:28.883385 32605 log.cc:824] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:28.885448 32605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1: No bootstrap required, opened a new log
I20260811 22:21:28.887238 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:28.874471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.26.125.62:41427)
0811 22:21:28.875003 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:28.875031 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:28.875073 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:21:28.875786 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:28.875805 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:28.876167 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:28.876505 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:28.877320 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:28.878316 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:28.879267 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:28.879759 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:28.879907 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:28.882565 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:28.882572 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:28.882588 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:28.882624 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:28.884286 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:28.884729 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:28.884735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:28.884741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:28.884857 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:28.885130 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:28.885133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:28.885134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:28.885581 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:28.885776 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:28.886068 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 22:21:28.886380 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":157,"threads_started":1}
I20260811 22:21:28.889180 32605 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3050063ee446b4af02d8ea559720b1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41427 } }
I20260811 22:21:28.889459 32605 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:28.889580 32605 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf3050063ee446b4af02d8ea559720b1, State: Initialized, Role: FOLLOWER
I20260811 22:21:28.890029 32605 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3050063ee446b4af02d8ea559720b1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41427 } }
I20260811 22:21:28.890182 32605 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:28.890228 32605 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:28.890300 32605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:28.891301 32605 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3050063ee446b4af02d8ea559720b1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41427 } }
I20260811 22:21:28.891633 32605 leader_election.cc:302] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf3050063ee446b4af02d8ea559720b1; no voters:
I20260811 22:21:28.891930 32605 leader_election.cc:288] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:28.892062 32615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:28.892290 32615 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [term 1 LEADER]: Becoming Leader. State: Replica: cf3050063ee446b4af02d8ea559720b1, State: Running, Role: LEADER
I20260811 22:21:28.892675 32615 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3050063ee446b4af02d8ea559720b1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41427 } }
I20260811 22:21:28.893240 32605 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:28.894840 32617 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf3050063ee446b4af02d8ea559720b1. Latest consensus state: current_term: 1 leader_uuid: "cf3050063ee446b4af02d8ea559720b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3050063ee446b4af02d8ea559720b1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41427 } } }
I20260811 22:21:28.895077 32617 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:28.894840 32616 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf3050063ee446b4af02d8ea559720b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3050063ee446b4af02d8ea559720b1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41427 } } }
I20260811 22:21:28.895166 32616 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:28.895470 32634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:28.898953 32634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:28.905700 32634 catalog_manager.cc:1347] Generated new cluster ID: cc84dc20917b4833a6206397ed777788
I20260811 22:21:28.905802 32634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:28.926229 32634 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:28.927670 32634 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:28.934971 32634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf3050063ee446b4af02d8ea559720b1: Generated new TSK 0
I20260811 22:21:28.935957 32634 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:28.979948 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41427
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:29.105296 32651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:29.105647 32651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:29.105696 32651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:29.109908 32651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:29.110000 32651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:29.110066 32651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:29.116146 32651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:29.117347 32651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:29.118463 32651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:29.129304 32660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:29.129302 32651 server_base.cc:1034] running on GCE node
W20260811 22:21:29.129271 32661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:29.129547 32669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:29.130066 32651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:29.130927 32651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:29.132125 32651 hybrid_clock.cc:648] HybridClock initialized: now 1786486889132100 us; error 40 us; skew 500 ppm
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486889, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:29.135267 32651 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:29.136423 32651 webserver.cc:460] Webserver started at http://127.26.125.1:42739/ using document root <none> and password file <none>
I20260811 22:21:29.137063 32651 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:29.137144 32651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:29.137383 32651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:29.139709 32651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "0409996b9cf941a690ef7d67ef46ad4e"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "39361987b3865794d63c30aa4a4783b2"
server_key_iv: "117299234d3a456dd85aa8af447c134f"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.140304 32651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "0409996b9cf941a690ef7d67ef46ad4e"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "39361987b3865794d63c30aa4a4783b2"
server_key_iv: "117299234d3a456dd85aa8af447c134f"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.144399 32651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:29.147131 32682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.148674 32651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:29.148811 32651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "0409996b9cf941a690ef7d67ef46ad4e"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "39361987b3865794d63c30aa4a4783b2"
server_key_iv: "117299234d3a456dd85aa8af447c134f"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.148934 32651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:29.173940 32651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:29.177505 32651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:29.177711 32651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:29.178363 32651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:29.179374 32651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:29.179435 32651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.179497 32651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:29.179522 32651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.188601 32651 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:38631
I20260811 22:21:29.188678 421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:38631 every 8 connection(s)
I20260811 22:21:29.189633 32651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:29.190198 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 32651
I20260811 22:21:29.190325 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:29.190646 27124 external_mini_cluster.cc:1428] Setting key 131c33ad99ac7dbefc161a80606da998
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486889, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:21:29.205406 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:29.191657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:50279 (local address 127.26.125.62:41427)
0811 22:21:29.191794 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:29.191798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:29.192989 (+ 1191us) server_negotiation.cc:407] Connection header received
0811 22:21:29.194844 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:29.194871 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:29.194941 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:29.195017 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:29.196289 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.196829 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.198025 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.198279 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.198369 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:29.201065 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:29.201069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:29.201071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:29.201097 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:29.202577 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:29.203407 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:29.203410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:29.203412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:29.203556 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:29.204035 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:29.204039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:29.204041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:29.204328 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:29.204448 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:29.205058 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 22:21:29.205221 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:21:29.206283 429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:29.192180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41427 (local address 127.26.125.1:50279)
0811 22:21:29.192820 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:29.192886 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:29.194595 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:29.195210 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:29.195218 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:29.195629 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:29.196141 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:29.196156 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.196952 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.196955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:29.197902 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:29.197912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.198459 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.198464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:29.198649 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:29.199252 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:29.199270 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:29.200869 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:29.202709 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:29.202716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:29.202731 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:29.203240 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:29.203680 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:29.203684 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:29.203686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:29.203919 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:29.204467 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:29.204474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:29.204778 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 22:21:29.204989 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:21:29.205298 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":133,"threads_started":1}
I20260811 22:21:29.207898 423 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41427
I20260811 22:21:29.208189 423 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:29.208843 423 heartbeater.cc:511] Master 127.26.125.62:41427 requested a full tablet report, sending...
I20260811 22:21:29.210830 32541 ts_manager.cc:194] Registered new tserver with Master: 0409996b9cf941a690ef7d67ef46ad4e (127.26.125.1:38631)
I20260811 22:21:29.212141 32541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:50279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:29.223578 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41427
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:29.349789 433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:29.350136 433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:29.350216 433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:29.354612 433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:29.354707 433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:29.354791 433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:29.360697 433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:29.361931 433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:29.363119 433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:29.373924 447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:29.373915 442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:29.374006 433 server_base.cc:1034] running on GCE node
W20260811 22:21:29.373915 445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:29.374763 433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:29.375618 433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:29.376825 433 hybrid_clock.cc:648] HybridClock initialized: now 1786486889376782 us; error 59 us; skew 500 ppm
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486889, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:29.380093 433 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:29.381130 433 webserver.cc:460] Webserver started at http://127.26.125.2:45225/ using document root <none> and password file <none>
I20260811 22:21:29.381843 433 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:29.381915 433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:29.382134 433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:29.384582 433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "aa5e7fefdbdb429a945de97f722834d3"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "918bf29c83bb39caec62d695a80be026"
server_key_iv: "84454c6fb6de5e12ee7f29aadffb4a35"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.385250 433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "aa5e7fefdbdb429a945de97f722834d3"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "918bf29c83bb39caec62d695a80be026"
server_key_iv: "84454c6fb6de5e12ee7f29aadffb4a35"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.389904 433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:29.393051 456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.394714 433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:29.394840 433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "aa5e7fefdbdb429a945de97f722834d3"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "918bf29c83bb39caec62d695a80be026"
server_key_iv: "84454c6fb6de5e12ee7f29aadffb4a35"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.394992 433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:29.426751 433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:29.430270 433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:29.430467 433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:29.431170 433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:29.432071 433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:29.432120 433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.432163 433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:29.432192 433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.441151 433 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:34975
I20260811 22:21:29.441174 647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:34975 every 8 connection(s)
I20260811 22:21:29.442190 433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:29.442783 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 433
I20260811 22:21:29.442934 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:29.443224 27124 external_mini_cluster.cc:1428] Setting key bba1d8b6a99113e0c648fcbf8221ca0c
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486889, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:21:29.458488 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:29.444219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:55995 (local address 127.26.125.62:41427)
0811 22:21:29.444362 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:29.444366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:29.445441 (+ 1075us) server_negotiation.cc:407] Connection header received
0811 22:21:29.446907 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:29.446910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:29.446961 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:29.447036 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:29.448710 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.449265 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.451034 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.451265 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.451348 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:29.454574 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:29.454578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:29.454580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:29.454609 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:29.456114 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:29.456702 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:29.456707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:29.456708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:29.456800 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:29.457168 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:29.457170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:29.457172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:29.457421 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:29.457504 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:29.458144 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 22:21:29.458298 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:21:29.459463 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:29.444733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41427 (local address 127.26.125.2:55995)
0811 22:21:29.445284 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:29.445332 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:29.446628 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:29.447261 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:29.447270 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:29.447787 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:29.448506 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:29.448526 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.449404 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.449407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:29.450902 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:29.450915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.451427 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.451432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:29.451648 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:29.452301 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:29.452324 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:29.454353 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:29.456230 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:29.456237 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:29.456253 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:29.456573 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:29.456891 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:29.456894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:29.456895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:29.457084 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:29.457522 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:29.457529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:29.457856 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 22:21:29.458074 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 22:21:29.458460 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":160,"threads_started":1}
I20260811 22:21:29.460708 648 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41427
I20260811 22:21:29.460963 648 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:29.461505 648 heartbeater.cc:511] Master 127.26.125.62:41427 requested a full tablet report, sending...
I20260811 22:21:29.462639 32541 ts_manager.cc:194] Registered new tserver with Master: aa5e7fefdbdb429a945de97f722834d3 (127.26.125.2:34975)
I20260811 22:21:29.463487 32541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:55995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:29.475916 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41427
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:29.600607 654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:29.600935 654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:29.600983 654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:29.605202 654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:29.605278 654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:29.605338 654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:29.610950 654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:29.612079 654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:29.613148 654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:29.624485 654 server_base.cc:1034] running on GCE node
W20260811 22:21:29.624333 664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:29.624338 665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:29.624333 667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:29.625270 654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:29.626010 654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:29.627188 654 hybrid_clock.cc:648] HybridClock initialized: now 1786486889627166 us; error 41 us; skew 500 ppm
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486889, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:29.630434 654 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:29.631481 654 webserver.cc:460] Webserver started at http://127.26.125.3:41141/ using document root <none> and password file <none>
I20260811 22:21:29.632076 654 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:29.632143 654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:29.632331 654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:29.634716 654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "ccf146c5b6244186af53d03e9ef6c41f"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "efbfa54e829725a354b7c5c96abd2eb3"
server_key_iv: "a6240c68eba8598d33a3bc71e019111a"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.635353 654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "ccf146c5b6244186af53d03e9ef6c41f"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "efbfa54e829725a354b7c5c96abd2eb3"
server_key_iv: "a6240c68eba8598d33a3bc71e019111a"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.639442 654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:29.642149 677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.643685 654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:29.643814 654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "ccf146c5b6244186af53d03e9ef6c41f"
format_stamp: "Formatted at 2026-08-11 22:21:29 on dist-test-slave-5bn2"
server_key: "efbfa54e829725a354b7c5c96abd2eb3"
server_key_iv: "a6240c68eba8598d33a3bc71e019111a"
server_key_version: "encryptionkey@0"
I20260811 22:21:29.643956 654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:29.659327 654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:29.662900 654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:29.663106 654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:29.663743 654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:29.664742 654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:29.664796 654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.664846 654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:29.664868 654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:29.673775 654 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:33633
I20260811 22:21:29.673880 895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:33633 every 8 connection(s)
I20260811 22:21:29.675010 654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:29.684793 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 654
I20260811 22:21:29.684924 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:29.685238 27124 external_mini_cluster.cc:1428] Setting key c5958f64a8bd0f897e9defe340970499
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486889, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:29.691118 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:29.677286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:54457 (local address 127.26.125.62:41427)
0811 22:21:29.677434 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:29.677438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:29.678417 (+ 979us) server_negotiation.cc:407] Connection header received
0811 22:21:29.679938 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:29.679942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:29.679999 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:29.680080 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:29.681562 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.682114 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.683511 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.683749 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.683850 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:29.687196 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:29.687201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:29.687205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:29.687240 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:29.688801 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:29.689325 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:29.689328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:29.689329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:29.689413 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:29.689854 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:29.689857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:29.689858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:29.690068 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:29.690159 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:29.690779 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 22:21:29.690973 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:21:29.692049 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:29.677729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41427 (local address 127.26.125.3:54457)
0811 22:21:29.678239 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:29.678290 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:29.679666 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:29.680336 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:29.680346 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:29.680773 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:29.681335 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:29.681350 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.682259 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.682262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:29.683351 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:29.683370 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.683954 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.683958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:29.684147 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:29.684961 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:29.684989 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:29.686951 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:29.688919 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:29.688927 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:29.688945 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:29.689227 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:29.689568 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:29.689571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:29.689573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:29.689745 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:29.690187 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:29.690194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:29.690491 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 22:21:29.690669 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 22:21:29.691050 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":126,"threads_started":1}
I20260811 22:21:29.693432 896 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41427
I20260811 22:21:29.693768 896 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:29.694442 896 heartbeater.cc:511] Master 127.26.125.62:41427 requested a full tablet report, sending...
I20260811 22:21:29.695704 32541 ts_manager.cc:194] Registered new tserver with Master: ccf146c5b6244186af53d03e9ef6c41f (127.26.125.3:33633)
I20260811 22:21:29.696275 32541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:54457
I20260811 22:21:29.700428 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486889, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486889, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:29.729457 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:29.720312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.26.125.62:41427)
0811 22:21:29.720484 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:29.720489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:29.720620 (+ 131us) server_negotiation.cc:407] Connection header received
0811 22:21:29.720771 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:29.720774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:29.720822 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:29.720892 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:29.721633 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.722186 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.723071 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:29.723307 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:29.723398 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:29.726022 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:29.726025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:29.726027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:29.726059 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:29.727628 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:29.728104 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:29.728107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:29.728108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:29.728195 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:29.728463 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:29.728466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:29.728468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:29.728760 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:29.728884 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:29.729164 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 22:21:29.729288 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 22:21:29.729903 32541 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42450
I20260811 22:21:29.730789 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 32651
I20260811 22:21:29.750303 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 433
I20260811 22:21:29.791360 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 654
I20260811 22:21:29.807502 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 32472
2026-08-11T22:21:29Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2479 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[914](info): set up 2 sockets
Aug 11 22:21:29 dist-test-slave-5bn2 krb5kdc[914](info): commencing operation
krb5kdc: starting...
W20260811 22:21:31.044667 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:31 dist-test-slave-5bn2 krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:31.180250 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45113
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:45113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:31.297204 941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:31.297530 941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:31.297595 941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:31.301729 941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:31.301820 941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:31.301848 941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:31.301868 941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:31.301889 941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:31.307614 941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:45113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:31.308876 941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:31.310014 941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:31.319463 941 server_base.cc:1034] running on GCE node
W20260811 22:21:31.319397 952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:31.319397 955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:31.319397 953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:31.320408 941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:31.321625 941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:31.322808 941 hybrid_clock.cc:648] HybridClock initialized: now 1786486891322787 us; error 34 us; skew 500 ppm
Aug 11 22:21:31 dist-test-slave-5bn2 krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:31.325977 941 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:31.327015 941 webserver.cc:460] Webserver started at http://127.26.125.62:35967/ using document root <none> and password file <none>
I20260811 22:21:31.327603 941 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:31.327667 941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:31.327898 941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:31.330140 941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "4681e035aa8447af980fe56e325935d2"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "da71afb53a02483a30fd62520387469e"
server_key_iv: "b96c69ab2c5232970449ec486fbf571e"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.330765 941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "4681e035aa8447af980fe56e325935d2"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "da71afb53a02483a30fd62520387469e"
server_key_iv: "b96c69ab2c5232970449ec486fbf571e"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.335021 941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:21:31.337775 968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:31.339207 941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:31.339341 941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "4681e035aa8447af980fe56e325935d2"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "da71afb53a02483a30fd62520387469e"
server_key_iv: "b96c69ab2c5232970449ec486fbf571e"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.339457 941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:31.362483 941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:31.366065 941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:31.366262 941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:31.373507 941 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:45113
I20260811 22:21:31.373569 1066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:45113 every 8 connection(s)
I20260811 22:21:31.374557 941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:31.377525 1068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:31.378880 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 941
I20260811 22:21:31.379014 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:31.379318 27124 external_mini_cluster.cc:1428] Setting key f05b859f102862101ad7487829ad6cb4
I20260811 22:21:31.384270 1068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2: Bootstrap starting.
I20260811 22:21:31.386912 1068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:31.387998 1068 log.cc:824] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:21:31 dist-test-slave-5bn2 krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:31.390501 1068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2: No bootstrap required, opened a new log
I20260811 22:21:31.394332 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:31.380927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40464 (local address 127.26.125.62:45113)
0811 22:21:31.381467 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:31.381485 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:31.381526 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:21:31.382191 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:31.382211 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:31.382585 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:31.382970 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:31.383891 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.384956 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.385829 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.386338 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.386501 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:31.389468 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:31.389473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:31.389521 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:31.389552 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:31.391323 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:31.391791 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:31.391797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:31.391803 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:31.391939 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:31.392249 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:31.392252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:31.392253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:31.392700 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:31.392886 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:31.393196 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 22:21:31.393521 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":143,"threads_started":1}
I20260811 22:21:31.394294 1068 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4681e035aa8447af980fe56e325935d2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45113 } }
I20260811 22:21:31.394608 1068 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:31.394678 1068 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4681e035aa8447af980fe56e325935d2, State: Initialized, Role: FOLLOWER
I20260811 22:21:31.395213 1068 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4681e035aa8447af980fe56e325935d2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45113 } }
I20260811 22:21:31.395363 1068 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:31.395431 1068 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:31.395543 1068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:31.396822 1068 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4681e035aa8447af980fe56e325935d2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45113 } }
I20260811 22:21:31.397210 1068 leader_election.cc:302] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4681e035aa8447af980fe56e325935d2; no voters:
I20260811 22:21:31.397522 1068 leader_election.cc:288] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:31.397627 1075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:31.397843 1075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [term 1 LEADER]: Becoming Leader. State: Replica: 4681e035aa8447af980fe56e325935d2, State: Running, Role: LEADER
I20260811 22:21:31.398192 1075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4681e035aa8447af980fe56e325935d2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45113 } }
I20260811 22:21:31.398900 1068 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:31.400056 1077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4681e035aa8447af980fe56e325935d2. Latest consensus state: current_term: 1 leader_uuid: "4681e035aa8447af980fe56e325935d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4681e035aa8447af980fe56e325935d2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45113 } } }
I20260811 22:21:31.400223 1077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:31.400056 1076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4681e035aa8447af980fe56e325935d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4681e035aa8447af980fe56e325935d2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45113 } } }
I20260811 22:21:31.400488 1076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:31.400671 1092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:31.404058 1092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:31.409940 1092 catalog_manager.cc:1347] Generated new cluster ID: 6dbe58ccb46d4b10b807c6c4c9677765
I20260811 22:21:31.410025 1092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:31.445994 1092 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:31.447460 1092 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:31.455554 1092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4681e035aa8447af980fe56e325935d2: Generated new TSK 0
I20260811 22:21:31.456313 1092 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:31.497753 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45113
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:31.615664 1113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:31.615968 1113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:31.616032 1113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:31.620471 1113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:31.620553 1113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:31.620636 1113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:31.626103 1113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:31.627257 1113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:31.628328 1113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:31.638489 1121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:31.638517 1120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:31.638564 1113 server_base.cc:1034] running on GCE node
W20260811 22:21:31.638506 1123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:31.639174 1113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:31.639815 1113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:31.640969 1113 hybrid_clock.cc:648] HybridClock initialized: now 1786486891640950 us; error 35 us; skew 500 ppm
Aug 11 22:21:31 dist-test-slave-5bn2 krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:31.643915 1113 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:31.644904 1113 webserver.cc:460] Webserver started at http://127.26.125.1:36745/ using document root <none> and password file <none>
I20260811 22:21:31.645481 1113 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:31.645581 1113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:31.645792 1113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:31.647945 1113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "23a7e0b6efb94e52a933fcb4c87999c2"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "8acb41b86943a9c54242fae7b9d90c58"
server_key_iv: "37dcaa3a8224a11215d40160f5bd99c6"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.648516 1113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "23a7e0b6efb94e52a933fcb4c87999c2"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "8acb41b86943a9c54242fae7b9d90c58"
server_key_iv: "37dcaa3a8224a11215d40160f5bd99c6"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.652456 1113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:31.655184 1136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:31.656600 1113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:21:31.656724 1113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "23a7e0b6efb94e52a933fcb4c87999c2"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "8acb41b86943a9c54242fae7b9d90c58"
server_key_iv: "37dcaa3a8224a11215d40160f5bd99c6"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.656833 1113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:31.677963 1113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:31.681663 1113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:31.681886 1113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:31.682693 1113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:31.683779 1113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:31.683887 1113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:31.683969 1113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:31.684000 1113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:31.692772 1113 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:39621
I20260811 22:21:31.692809 1310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:39621 every 8 connection(s)
I20260811 22:21:31.693738 1113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:31.696132 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1113
I20260811 22:21:31.696270 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:31.696619 27124 external_mini_cluster.cc:1428] Setting key a0e16b92436983ef6868d0cd93f32672
Aug 11 22:21:31 dist-test-slave-5bn2 krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:21:31.709591 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:31.695770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:41321 (local address 127.26.125.62:45113)
0811 22:21:31.695947 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:31.695951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:31.697016 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 22:21:31.698550 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:31.698553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:31.698608 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:31.698688 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:31.700147 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.700747 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.702141 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.702472 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.702580 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:31.705583 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:31.705587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:31.705595 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:31.705623 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:31.707144 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:31.707799 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:31.707803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:31.707804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:31.707890 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:31.708265 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:31.708269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:31.708270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:31.708572 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:31.708669 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:31.709246 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 22:21:31.709377 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:21:31.710695 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:31.696268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45113 (local address 127.26.125.1:41321)
0811 22:21:31.696860 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:31.696907 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:31.698331 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:31.698920 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:31.698928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:31.699330 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:31.699938 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:31.699960 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.700880 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.700884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:31.702013 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:31.702023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.702659 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.702663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:31.702838 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:31.703608 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:31.703627 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:31.705344 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:31.707276 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:31.707284 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:31.707307 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:31.707701 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:31.707998 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:31.708000 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:31.708002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:31.708155 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:31.708681 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:31.708687 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:31.708996 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 22:21:31.709175 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 22:21:31.709494 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":143,"threads_started":1}
I20260811 22:21:31.712131 1319 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45113
I20260811 22:21:31.712494 1319 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:31.713055 1319 heartbeater.cc:511] Master 127.26.125.62:45113 requested a full tablet report, sending...
I20260811 22:21:31.714699 1000 ts_manager.cc:194] Registered new tserver with Master: 23a7e0b6efb94e52a933fcb4c87999c2 (127.26.125.1:39621)
I20260811 22:21:31.715952 1000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:41321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:31.726840 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45113
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:31.844069 1331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:31.844370 1331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:31.844435 1331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:31.848582 1331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:31.848659 1331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:31.848744 1331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:31.854466 1331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:31.855738 1331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:31.856834 1331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:31.866753 1344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:31.866842 1347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:31.866771 1343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:31.867127 1331 server_base.cc:1034] running on GCE node
I20260811 22:21:31.867537 1331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:31.868209 1331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:31.869421 1331 hybrid_clock.cc:648] HybridClock initialized: now 1786486891869388 us; error 42 us; skew 500 ppm
Aug 11 22:21:31 dist-test-slave-5bn2 krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:31.872365 1331 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:31.873379 1331 webserver.cc:460] Webserver started at http://127.26.125.2:40283/ using document root <none> and password file <none>
I20260811 22:21:31.874020 1331 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:31.874096 1331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:31.874320 1331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:31.876564 1331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "62a75d8c5d214f43937fd8abe705d760"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "22989a429e84034b4d4e2e0140c5d34c"
server_key_iv: "5da22535437c9e1c5d202f52c7cdf4a8"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.877159 1331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "62a75d8c5d214f43937fd8abe705d760"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "22989a429e84034b4d4e2e0140c5d34c"
server_key_iv: "5da22535437c9e1c5d202f52c7cdf4a8"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.884688 1331 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 22:21:31.888564 1355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:31.890059 1331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:31.890200 1331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "62a75d8c5d214f43937fd8abe705d760"
format_stamp: "Formatted at 2026-08-11 22:21:31 on dist-test-slave-5bn2"
server_key: "22989a429e84034b4d4e2e0140c5d34c"
server_key_iv: "5da22535437c9e1c5d202f52c7cdf4a8"
server_key_version: "encryptionkey@0"
I20260811 22:21:31.890311 1331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:31.901599 1331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:31.905146 1331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:31.905325 1331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:31.905920 1331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:31.906795 1331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:31.906844 1331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:31.906941 1331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:31.906972 1331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:31.915696 1331 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:46255
I20260811 22:21:31.915724 1558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:46255 every 8 connection(s)
I20260811 22:21:31.916703 1331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:31.924659 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1331
I20260811 22:21:31.924769 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:31.925077 27124 external_mini_cluster.cc:1428] Setting key 08b2b068b4ae29616764042b6aeff966
Aug 11 22:21:31 dist-test-slave-5bn2 krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:31.931309 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:31.918777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:53339 (local address 127.26.125.62:45113)
0811 22:21:31.918939 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:31.918943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:31.919942 (+ 999us) server_negotiation.cc:407] Connection header received
0811 22:21:31.921413 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:31.921417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:31.921480 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:31.921556 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:31.922882 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.923427 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.924612 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.924869 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.924972 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:31.927609 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:31.927613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:31.927616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:31.927647 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:31.929112 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:31.929646 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:31.929649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:31.929651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:31.929735 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:31.930070 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:31.930073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:31.930075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:31.930315 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:31.930416 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:31.931001 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 22:21:31.931139 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:21:31.932394 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:31.919252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45113 (local address 127.26.125.2:53339)
0811 22:21:31.919743 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:31.919789 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:31.921170 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:31.921761 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:31.921769 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:31.922165 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:31.922692 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:31.922707 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.923566 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.923569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:31.924487 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:31.924497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:31.925024 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:31.925028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:31.925222 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:31.925845 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:31.925864 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:31.927427 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:31.929234 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:31.929241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:31.929262 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:31.929550 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:31.929818 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:31.929821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:31.929823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:31.929980 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:31.930425 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:31.930431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:31.930717 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 22:21:31.930943 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 22:21:31.931339 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":145,"threads_started":1}
I20260811 22:21:31.933656 1559 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45113
I20260811 22:21:31.933923 1559 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:31.934466 1559 heartbeater.cc:511] Master 127.26.125.62:45113 requested a full tablet report, sending...
I20260811 22:21:31.935665 1000 ts_manager.cc:194] Registered new tserver with Master: 62a75d8c5d214f43937fd8abe705d760 (127.26.125.2:46255)
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:31.936394 1000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:53339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:31.956264 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45113
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:32.077726 1573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:32.078053 1573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:32.078117 1573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:32.082436 1573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:32.082545 1573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:32.082630 1573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:32.088413 1573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:32.089675 1573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:32.090818 1573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:32.101132 1579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:32.101135 1586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:32.101199 1583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:32.101387 1573 server_base.cc:1034] running on GCE node
I20260811 22:21:32.101939 1573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:32.102641 1573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:32.103822 1573 hybrid_clock.cc:648] HybridClock initialized: now 1786486892103778 us; error 62 us; skew 500 ppm
Aug 11 22:21:32 dist-test-slave-5bn2 krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:32.106928 1573 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:32.107960 1573 webserver.cc:460] Webserver started at http://127.26.125.3:35911/ using document root <none> and password file <none>
I20260811 22:21:32.108562 1573 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:32.108644 1573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:32.108862 1573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:32.111174 1573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "ff465e86c1264539aff5252517f99b62"
format_stamp: "Formatted at 2026-08-11 22:21:32 on dist-test-slave-5bn2"
server_key: "743189df3015c4e522c6bb66b1e722e5"
server_key_iv: "81a378087c3395ac2bde6cff94b6bead"
server_key_version: "encryptionkey@0"
I20260811 22:21:32.111749 1573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "ff465e86c1264539aff5252517f99b62"
format_stamp: "Formatted at 2026-08-11 22:21:32 on dist-test-slave-5bn2"
server_key: "743189df3015c4e522c6bb66b1e722e5"
server_key_iv: "81a378087c3395ac2bde6cff94b6bead"
server_key_version: "encryptionkey@0"
I20260811 22:21:32.115799 1573 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:32.118468 1601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:32.119887 1573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:32.120035 1573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "ff465e86c1264539aff5252517f99b62"
format_stamp: "Formatted at 2026-08-11 22:21:32 on dist-test-slave-5bn2"
server_key: "743189df3015c4e522c6bb66b1e722e5"
server_key_iv: "81a378087c3395ac2bde6cff94b6bead"
server_key_version: "encryptionkey@0"
I20260811 22:21:32.120143 1573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:32.139915 1573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:32.143492 1573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:32.143684 1573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:32.144380 1573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:32.145292 1573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:32.145342 1573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:32.145397 1573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:32.145427 1573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:32.154431 1573 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:44249
I20260811 22:21:32.154520 1804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:44249 every 8 connection(s)
I20260811 22:21:32.155427 1573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:32.164764 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1573
I20260811 22:21:32.164932 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:32.165228 27124 external_mini_cluster.cc:1428] Setting key 5e1ba3f51a3feecf08ec914c9bcd08cf
Aug 11 22:21:32 dist-test-slave-5bn2 krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486892, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:32.169667 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:32.157365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:37323 (local address 127.26.125.62:45113)
0811 22:21:32.157565 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:32.157568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:32.158521 (+ 953us) server_negotiation.cc:407] Connection header received
0811 22:21:32.160011 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:32.160014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:32.160071 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:32.160148 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:32.161459 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:32.162039 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:32.163242 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:32.163474 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:32.163566 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:32.166136 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:32.166139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:32.166142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:32.166170 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:32.167757 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:32.168270 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:32.168273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:32.168274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:32.168352 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:32.168679 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:32.168681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:32.168683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:32.168860 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:32.168935 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:32.169442 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 22:21:32.169546 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:21:32.170516 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:32.157796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45113 (local address 127.26.125.3:37323)
0811 22:21:32.158314 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:32.158360 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:32.159786 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:32.160347 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:32.160355 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:32.160777 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:32.161302 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:32.161318 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:32.162161 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:32.162164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:32.163126 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:32.163137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:32.163626 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:32.163630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:32.163793 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:32.164356 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:32.164372 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:32.165951 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:32.167880 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:32.167886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:32.167899 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:32.168180 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:32.168472 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:32.168474 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:32.168475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:32.168613 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:32.168935 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:32.168939 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:32.169194 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 22:21:32.169354 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:21:32.169655 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":131,"threads_started":1}
I20260811 22:21:32.171655 1806 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45113
I20260811 22:21:32.171875 1806 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:32.172329 1806 heartbeater.cc:511] Master 127.26.125.62:45113 requested a full tablet report, sending...
I20260811 22:21:32.173285 1000 ts_manager.cc:194] Registered new tserver with Master: ff465e86c1264539aff5252517f99b62 (127.26.125.3:44249)
I20260811 22:21:32.173849 1000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:37323
I20260811 22:21:32.179311 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:21:32 dist-test-slave-5bn2 krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:21:32 dist-test-slave-5bn2 krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:32.207232 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:32.197702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45744 (local address 127.26.125.1:39621)
0811 22:21:32.198042 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:32.198060 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:32.198088 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:21:32.198177 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:32.198183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:32.198386 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:32.198516 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:32.199339 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:32.200228 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:32.201134 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:32.201511 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:32.201601 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:32.203900 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:32.203910 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:32.203926 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:32.203952 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:32.205604 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:32.205978 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:32.205983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:32.205984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:32.206063 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:32.206249 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:32.206251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:32.206252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:32.206499 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:32.206696 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:32.206867 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:21:32.207024 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":115,"threads_started":1}
Aug 11 22:21:32 dist-test-slave-5bn2 krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:32.218173 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:32.210244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.26.125.62:45113)
0811 22:21:32.210380 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:32.210384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:32.210402 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:21:32.210540 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:32.210551 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:32.210598 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:32.210662 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:32.211326 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:32.211851 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:32.212603 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:32.212830 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:32.212916 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:32.215335 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:32.215339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:32.215341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:32.215368 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:32.216573 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:32.216955 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:32.216959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:32.216961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:32.217040 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:32.217281 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:32.217284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:32.217286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:32.217526 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:32.217657 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:32.217891 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 22:21:32.218012 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 22:21:32.218606 1000 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40478
I20260811 22:21:32.219269 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1113
I20260811 22:21:32.257228 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1331
I20260811 22:21:32.275331 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1573
I20260811 22:21:32.291445 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 941
2026-08-11T22:21:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2480 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:32 dist-test-slave-5bn2 krb5kdc[1829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:32 dist-test-slave-5bn2 krb5kdc[1829](info): set up 2 sockets
Aug 11 22:21:32 dist-test-slave-5bn2 krb5kdc[1829](info): commencing operation
krb5kdc: starting...
W20260811 22:21:33.491531 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:33 dist-test-slave-5bn2 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:33.625627 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:43429
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:33.742540 1850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:33.742838 1850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:33.742928 1850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:33.746948 1850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:33.747015 1850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:33.747048 1850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:33.747069 1850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:33.747094 1850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:33.752444 1850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:43429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:43429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:33.753548 1850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:33.754623 1850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:33.763110 1862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:33.763180 1850 server_base.cc:1034] running on GCE node
W20260811 22:21:33.763121 1865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:33.763129 1858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:33.763985 1850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:33.765120 1850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:33.766289 1850 hybrid_clock.cc:648] HybridClock initialized: now 1786486893766266 us; error 37 us; skew 500 ppm
Aug 11 22:21:33 dist-test-slave-5bn2 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:33.769282 1850 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:33.770059 1850 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:33.770138 1850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:33.770330 1850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:33.772503 1850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "899c69412c2e4e1c8129946697e3bdf7"
format_stamp: "Formatted at 2026-08-11 22:21:33 on dist-test-slave-5bn2"
server_key: "3efc703ac50bcb66816a2c49c20de420"
server_key_iv: "85e0ffd0d8297d9732c7446be01db072"
server_key_version: "encryptionkey@0"
I20260811 22:21:33.773033 1850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "899c69412c2e4e1c8129946697e3bdf7"
format_stamp: "Formatted at 2026-08-11 22:21:33 on dist-test-slave-5bn2"
server_key: "3efc703ac50bcb66816a2c49c20de420"
server_key_iv: "85e0ffd0d8297d9732c7446be01db072"
server_key_version: "encryptionkey@0"
I20260811 22:21:33.776940 1850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 22:21:33.779598 1875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:33.781029 1850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:21:33.781167 1850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "899c69412c2e4e1c8129946697e3bdf7"
format_stamp: "Formatted at 2026-08-11 22:21:33 on dist-test-slave-5bn2"
server_key: "3efc703ac50bcb66816a2c49c20de420"
server_key_iv: "85e0ffd0d8297d9732c7446be01db072"
server_key_version: "encryptionkey@0"
I20260811 22:21:33.781278 1850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:33.804121 1850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:33.807736 1850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:33.807929 1850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:33.814980 1850 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:43429
I20260811 22:21:33.815229 1966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:43429 every 8 connection(s)
I20260811 22:21:33.815868 1850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:33.819082 1967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:33.823755 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1850
I20260811 22:21:33.823910 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:33.824193 27124 external_mini_cluster.cc:1428] Setting key 14d65a10ef21e14cab400663e827ce0a
I20260811 22:21:33.825996 1967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7: Bootstrap starting.
I20260811 22:21:33.828944 1967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:33.829907 1967 log.cc:824] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:33.832064 1967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7: No bootstrap required, opened a new log
Aug 11 22:21:33 dist-test-slave-5bn2 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:33.836385 1967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899c69412c2e4e1c8129946697e3bdf7" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43429 } }
I20260811 22:21:33.836697 1967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:33.836805 1967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 899c69412c2e4e1c8129946697e3bdf7, State: Initialized, Role: FOLLOWER
I20260811 22:21:33.837366 1967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899c69412c2e4e1c8129946697e3bdf7" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43429 } }
I20260811 22:21:33.837548 1967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:33.837654 1967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:33.837754 1967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:33.838941 1967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899c69412c2e4e1c8129946697e3bdf7" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43429 } }
I20260811 22:21:33.839306 1967 leader_election.cc:302] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 899c69412c2e4e1c8129946697e3bdf7; no voters:
I20260811 22:21:33.839632 1967 leader_election.cc:288] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:33.839742 1972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:33.839926 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:33.825839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50322 (local address 127.26.125.62:43429)
0811 22:21:33.826322 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:33.826343 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:33.826389 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:21:33.827295 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:33.827325 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:33.827806 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:33.828234 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:33.828961 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:33.830507 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:33.831437 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:33.831935 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:33.832091 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:33.834810 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:33.834816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:33.834831 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:33.834899 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:33.836639 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:33.837158 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:33.837164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:33.837171 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:33.837302 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:33.837611 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:33.837614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:33.837616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:33.838120 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:33.838283 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:33.838573 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 22:21:33.838897 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":150,"threads_started":1}
I20260811 22:21:33.839959 1972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [term 1 LEADER]: Becoming Leader. State: Replica: 899c69412c2e4e1c8129946697e3bdf7, State: Running, Role: LEADER
I20260811 22:21:33.840288 1972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899c69412c2e4e1c8129946697e3bdf7" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43429 } }
I20260811 22:21:33.840945 1967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:33.841634 1972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 899c69412c2e4e1c8129946697e3bdf7. Latest consensus state: current_term: 1 leader_uuid: "899c69412c2e4e1c8129946697e3bdf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899c69412c2e4e1c8129946697e3bdf7" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43429 } } }
I20260811 22:21:33.841846 1972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:33.842060 1974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "899c69412c2e4e1c8129946697e3bdf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899c69412c2e4e1c8129946697e3bdf7" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43429 } } }
I20260811 22:21:33.842164 1974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:33.842283 1990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:33.845587 1990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:33.851780 1990 catalog_manager.cc:1347] Generated new cluster ID: 130fdd0acffd4d808d9a0b9bd580f833
I20260811 22:21:33.851893 1990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:33.873448 1990 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:33.874797 1990 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:33.881233 1990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7: Generated new TSK 0
I20260811 22:21:33.882187 1990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:33.918139 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43429
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:34.039543 2009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:34.039971 2009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:34.040045 2009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:34.044682 2009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:34.044767 2009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:34.044862 2009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:34.050451 2009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:34.051708 2009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:34.052820 2009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:34.063290 2021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:34.063228 2018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:34.063269 2009 server_base.cc:1034] running on GCE node
W20260811 22:21:34.063228 2017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:34.064049 2009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:34.064810 2009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:34.065990 2009 hybrid_clock.cc:648] HybridClock initialized: now 1786486894065970 us; error 40 us; skew 500 ppm
Aug 11 22:21:34 dist-test-slave-5bn2 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:34.069185 2009 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:34.070292 2009 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:34.070401 2009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:34.070664 2009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:34.073673 2009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "130e32572fab495c823e3b4efce862e3"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "66692af4c43c12b0336e3db06993aa41"
server_key_iv: "90c413c992f36c02b82aceae0e7acce0"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.074447 2009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "130e32572fab495c823e3b4efce862e3"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "66692af4c43c12b0336e3db06993aa41"
server_key_iv: "90c413c992f36c02b82aceae0e7acce0"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.080608 2009 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.003s
I20260811 22:21:34.083788 2030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.085310 2009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:21:34.085474 2009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "130e32572fab495c823e3b4efce862e3"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "66692af4c43c12b0336e3db06993aa41"
server_key_iv: "90c413c992f36c02b82aceae0e7acce0"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.085618 2009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:34.102207 2009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:34.105754 2009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:34.105964 2009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:34.106602 2009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:34.107578 2009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:34.107630 2009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.107688 2009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:34.107713 2009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.116482 2009 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:46879
I20260811 22:21:34.116539 2234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:46879 every 8 connection(s)
I20260811 22:21:34.117137 2009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:34.119963 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 2009
I20260811 22:21:34.120105 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:34.120436 27124 external_mini_cluster.cc:1428] Setting key 4c4300deee16389a1944179a43b9806b
Aug 11 22:21:34 dist-test-slave-5bn2 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:21:34.132318 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.119018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:35559 (local address 127.26.125.62:43429)
0811 22:21:34.119158 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.119162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.120208 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 22:21:34.121792 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.121795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.121838 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.121933 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:34.123262 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.123799 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.124966 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.125193 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.125283 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.128162 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:34.128167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:34.128170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:34.128205 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:34.129931 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:34.130589 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:34.130593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:34.130595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:34.130696 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:34.131091 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:34.131095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:34.131097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:34.131362 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:34.131474 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.132031 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.132181 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:21:34.133224 2237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.119477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:43429 (local address 127.26.125.1:35559)
0811 22:21:34.120033 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.120100 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:34.121559 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:34.122131 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:34.122139 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:34.122525 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:34.123111 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.123130 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.123932 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.123935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.124835 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.124844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.125337 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.125341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.125552 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.126185 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:34.126206 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:34.127962 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:34.130072 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:34.130091 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:34.130108 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:34.130466 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:34.130784 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:34.130786 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:34.130787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:34.130987 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:34.131477 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:34.131483 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:34.131755 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 22:21:34.131943 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 22:21:34.132252 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:21:34.134699 2235 heartbeater.cc:347] Connected to a master server at 127.26.125.62:43429
I20260811 22:21:34.135002 2235 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:34.135617 2235 heartbeater.cc:511] Master 127.26.125.62:43429 requested a full tablet report, sending...
I20260811 22:21:34.137458 1907 ts_manager.cc:194] Registered new tserver with Master: 130e32572fab495c823e3b4efce862e3 (127.26.125.1:46879)
I20260811 22:21:34.138644 1907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:35559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:34.151868 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43429
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:34.271052 2242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:34.271407 2242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:34.271456 2242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:34.275781 2242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:34.275870 2242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:34.275945 2242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:34.282015 2242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:34.283447 2242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:34.284545 2242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:34.295117 2252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:34.295171 2251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:34.295117 2257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:34.295311 2242 server_base.cc:1034] running on GCE node
I20260811 22:21:34.296013 2242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:34.296669 2242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:34.297864 2242 hybrid_clock.cc:648] HybridClock initialized: now 1786486894297828 us; error 53 us; skew 500 ppm
Aug 11 22:21:34 dist-test-slave-5bn2 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:34.300933 2242 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:34.301754 2242 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:34.301831 2242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:34.302044 2242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:34.304322 2242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "53a09e83f756415b9dcb3a71ba6388f9"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "61bcb68cc973fdff05ba53c20072ffe6"
server_key_iv: "90620dc35eac7bea5ee51d372ebd7304"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.304921 2242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "53a09e83f756415b9dcb3a71ba6388f9"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "61bcb68cc973fdff05ba53c20072ffe6"
server_key_iv: "90620dc35eac7bea5ee51d372ebd7304"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.309315 2242 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:34.312462 2267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.313935 2242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:34.314083 2242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "53a09e83f756415b9dcb3a71ba6388f9"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "61bcb68cc973fdff05ba53c20072ffe6"
server_key_iv: "90620dc35eac7bea5ee51d372ebd7304"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.314208 2242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:34.327338 2242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:34.330717 2242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:34.330966 2242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:34.331589 2242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:34.332458 2242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:34.332506 2242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.332571 2242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:34.332597 2242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.341465 2242 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:35645
I20260811 22:21:34.341542 2455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:35645 every 8 connection(s)
I20260811 22:21:34.342125 2242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:34.349972 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 2242
I20260811 22:21:34.350124 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:34.350407 27124 external_mini_cluster.cc:1428] Setting key 4b969ca6e359d7d52f9079e82a58d5cc
Aug 11 22:21:34 dist-test-slave-5bn2 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:34.357167 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.344088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:48637 (local address 127.26.125.62:43429)
0811 22:21:34.344225 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.344228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.345218 (+ 990us) server_negotiation.cc:407] Connection header received
0811 22:21:34.346579 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.346582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.346632 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.346708 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:34.347985 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.348518 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.349699 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.350002 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.350135 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.353133 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:34.353136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:34.353138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:34.353164 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:34.354899 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:34.355443 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:34.355446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:34.355448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:34.355549 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:34.355925 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:34.355928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:34.355929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:34.356130 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:34.356207 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.356915 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.357038 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:21:34.358433 2459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.344509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:43429 (local address 127.26.125.2:48637)
0811 22:21:34.345053 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.345101 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:34.346359 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:34.346930 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:34.346938 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:34.347322 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:34.347841 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.347855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.348631 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.348634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.349575 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.349584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.350221 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.350226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.350401 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.351115 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:34.351134 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:34.352946 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:34.355021 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:34.355028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:34.355046 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:34.355340 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:34.355637 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:34.355640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:34.355642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:34.355831 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:34.356230 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:34.356237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:34.356635 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 22:21:34.356862 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 22:21:34.357242 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":122,"threads_started":1}
I20260811 22:21:34.359762 2457 heartbeater.cc:347] Connected to a master server at 127.26.125.62:43429
I20260811 22:21:34.360065 2457 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:34.360780 2457 heartbeater.cc:511] Master 127.26.125.62:43429 requested a full tablet report, sending...
I20260811 22:21:34.362012 1907 ts_manager.cc:194] Registered new tserver with Master: 53a09e83f756415b9dcb3a71ba6388f9 (127.26.125.2:35645)
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:34.362666 1907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:48637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:34.382539 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43429
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:34.501241 2468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:34.501565 2468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:34.501614 2468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:34.505862 2468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:34.505959 2468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:34.506024 2468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:34.511973 2468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:34.513226 2468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:34.514369 2468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:34.524490 2477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:34.524516 2480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:34.524516 2478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:34.524633 2468 server_base.cc:1034] running on GCE node
I20260811 22:21:34.525345 2468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:34.526000 2468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:34.527165 2468 hybrid_clock.cc:648] HybridClock initialized: now 1786486894527140 us; error 39 us; skew 500 ppm
Aug 11 22:21:34 dist-test-slave-5bn2 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:34.530220 2468 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:34.531173 2468 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:34.531255 2468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:34.531443 2468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:34.533649 2468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "2d06862741d2419f833315471a96de25"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "ec3ccdc945b1d2aa5d427a7a9a276e1b"
server_key_iv: "17a3e6979cc88478c75398498e8fd842"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.534225 2468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "2d06862741d2419f833315471a96de25"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "ec3ccdc945b1d2aa5d427a7a9a276e1b"
server_key_iv: "17a3e6979cc88478c75398498e8fd842"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.538337 2468 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:34.541127 2487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.542585 2468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:21:34.542711 2468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "2d06862741d2419f833315471a96de25"
format_stamp: "Formatted at 2026-08-11 22:21:34 on dist-test-slave-5bn2"
server_key: "ec3ccdc945b1d2aa5d427a7a9a276e1b"
server_key_iv: "17a3e6979cc88478c75398498e8fd842"
server_key_version: "encryptionkey@0"
I20260811 22:21:34.542830 2468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:34.577631 2468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:34.581120 2468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:34.581290 2468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:34.581898 2468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:34.582886 2468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:34.582947 2468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.582994 2468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:34.583014 2468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:34.592206 2468 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:43597
I20260811 22:21:34.592223 2694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:43597 every 8 connection(s)
I20260811 22:21:34.592854 2468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:34.601486 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 2468
I20260811 22:21:34.601644 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:34.601987 27124 external_mini_cluster.cc:1428] Setting key c616e7e36f9bf88077685050b00d4431
Aug 11 22:21:34 dist-test-slave-5bn2 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:34.607702 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.594866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:56977 (local address 127.26.125.62:43429)
0811 22:21:34.595020 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.595023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.595986 (+ 963us) server_negotiation.cc:407] Connection header received
0811 22:21:34.597341 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.597344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.597405 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.597491 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:34.598767 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.599357 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.600608 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.600866 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.600972 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.603840 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:34.603844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:34.603846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:34.603884 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:34.605320 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:34.605999 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:34.606002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:34.606004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:34.606093 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:34.606470 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:34.606473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:34.606475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:34.606714 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:34.606805 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.607424 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.607573 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:21:34.608610 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.595313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:43429 (local address 127.26.125.3:56977)
0811 22:21:34.595816 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.595865 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:34.597124 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:34.597699 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:34.597707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:34.598115 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:34.598623 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.598637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.599482 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.599485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.600372 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.600381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.601037 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.601041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.601247 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.601911 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:34.601933 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:34.603679 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:34.605499 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:34.605506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:34.605520 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:34.605892 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:34.606170 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:34.606172 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:34.606173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:34.606324 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:34.606821 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:34.606827 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:34.607161 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 22:21:34.607337 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 22:21:34.607684 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":123,"threads_started":1}
I20260811 22:21:34.609899 2695 heartbeater.cc:347] Connected to a master server at 127.26.125.62:43429
I20260811 22:21:34.610188 2695 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:34.610697 2695 heartbeater.cc:511] Master 127.26.125.62:43429 requested a full tablet report, sending...
I20260811 22:21:34.611817 1907 ts_manager.cc:194] Registered new tserver with Master: 2d06862741d2419f833315471a96de25 (127.26.125.3:43597)
I20260811 22:21:34.612378 1907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:56977
I20260811 22:21:34.615993 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:21:34.625259 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.618006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.26.125.62:43429)
0811 22:21:34.618134 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.618138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.618293 (+ 155us) server_negotiation.cc:407] Connection header received
0811 22:21:34.618404 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.618406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.618446 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.618511 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:34.619354 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.619898 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.620711 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.620916 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.620985 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.622294 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:34.622297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:34.622299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:34.622323 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:34.623695 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:34.624188 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:34.624190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:34.624191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:34.624260 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:34.624477 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:34.624479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:34.624480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:34.624721 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:34.624819 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.625037 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.625133 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:21:34.628247 1907 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:34.631420 1907 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:34.646485 2716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.640260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.26.125.2:35645)
0811 22:21:34.640752 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.640764 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.641084 (+ 320us) server_negotiation.cc:407] Connection header received
0811 22:21:34.641346 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.641352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.641577 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.641717 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:34.642322 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.643492 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.645049 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.645803 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.645917 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.646093 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.646223 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.646324 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":154,"threads_started":1}
I20260811 22:21:34.646481 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.640115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:35645 (local address 127.0.0.1:47724)
0811 22:21:34.640923 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.640978 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:34.641171 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:34.641970 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:34.641973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:34.641990 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:34.642167 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.642173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.643630 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.643633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.644930 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.644941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.645916 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.645919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.646018 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.646089 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 22:21:34.646191 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:21:34.646316 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":153,"thread_start_us":83,"threads_started":1}
I20260811 22:21:34.646910 2720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.640882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46879 (local address 127.0.0.1:48380)
0811 22:21:34.641236 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.641247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:34.641320 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:34.642406 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:34.642408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:34.642423 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:34.642489 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.642493 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.643580 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.643584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.645188 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.645199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.646217 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.646221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.646300 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.646321 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:21:34.646390 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:21:34.646444 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":286,"thread_start_us":127,"threads_started":1}
I20260811 22:21:34.647353 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.641472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.26.125.1:46879)
0811 22:21:34.641886 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.641892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.641914 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:21:34.641985 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.641990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.642163 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.642291 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:34.642603 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.643469 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.645315 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.646107 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.646207 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.646334 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.646574 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.647245 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":129,"threads_started":1}
I20260811 22:21:34.647971 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.640115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43597 (local address 127.0.0.1:35202)
0811 22:21:34.641004 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.641018 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:34.641157 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:34.641872 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:34.641878 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:34.641901 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:34.642106 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.642115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.643761 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.643763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.645188 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.645199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.647585 (+ 2386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.647588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.647677 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.647702 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:21:34.647772 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 22:21:34.647843 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":61,"spinlock_wait_cycles":9088,"thread_start_us":114,"threads_started":1}
I20260811 22:21:34.649853 2122 tablet_service.cc:1467] Processing CreateTablet for tablet 19b19ee939bd46d9bf902cfd7c75f418 (DEFAULT_TABLE table=test-table [id=68b9751ca14244608aae94f73815be73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:34.649797 2345 tablet_service.cc:1467] Processing CreateTablet for tablet 19b19ee939bd46d9bf902cfd7c75f418 (DEFAULT_TABLE table=test-table [id=68b9751ca14244608aae94f73815be73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:34.650796 2122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19b19ee939bd46d9bf902cfd7c75f418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:34.650796 2345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19b19ee939bd46d9bf902cfd7c75f418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:34.652632 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.640456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35202 (local address 127.26.125.3:43597)
0811 22:21:34.640920 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.640935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.641309 (+ 374us) server_negotiation.cc:407] Connection header received
0811 22:21:34.641396 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.641401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.641675 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.641824 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:34.642217 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.643507 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.646397 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.647323 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.647425 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.647566 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.652344 (+ 4778us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.652472 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":119,"threads_started":1}
I20260811 22:21:34.655822 2563 tablet_service.cc:1467] Processing CreateTablet for tablet 19b19ee939bd46d9bf902cfd7c75f418 (DEFAULT_TABLE table=test-table [id=68b9751ca14244608aae94f73815be73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:34.657032 2563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19b19ee939bd46d9bf902cfd7c75f418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:34.657883 2726 tablet_bootstrap.cc:492] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Bootstrap starting.
I20260811 22:21:34.658311 2727 tablet_bootstrap.cc:492] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3: Bootstrap starting.
I20260811 22:21:34.660837 2727 tablet_bootstrap.cc:654] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:34.661360 2726 tablet_bootstrap.cc:654] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:34.661839 2727 log.cc:824] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:34.662204 2726 log.cc:824] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:34.663828 2733 tablet_bootstrap.cc:492] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Bootstrap starting.
I20260811 22:21:34.664172 2727 tablet_bootstrap.cc:492] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3: No bootstrap required, opened a new log
I20260811 22:21:34.664388 2727 ts_tablet_manager.cc:1397] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:21:34.665854 2733 tablet_bootstrap.cc:654] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:34.667008 2733 log.cc:824] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:34.668062 2727 raft_consensus.cc:348] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
I20260811 22:21:34.668287 2727 raft_consensus.cc:374] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:34.668331 2727 raft_consensus.cc:729] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130e32572fab495c823e3b4efce862e3, State: Initialized, Role: FOLLOWER
I20260811 22:21:34.668516 2726 tablet_bootstrap.cc:492] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: No bootstrap required, opened a new log
I20260811 22:21:34.668736 2726 ts_tablet_manager.cc:1397] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260811 22:21:34.668769 2727 consensus_queue.cc:260] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
I20260811 22:21:34.669500 2235 heartbeater.cc:503] Master 127.26.125.62:43429 was elected leader, sending a full tablet report...
I20260811 22:21:34.669749 2727 ts_tablet_manager.cc:1428] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3: Time spent starting tablet: real 0.005s user 0.003s sys 0.004s
I20260811 22:21:34.671913 2733 tablet_bootstrap.cc:492] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: No bootstrap required, opened a new log
I20260811 22:21:34.672132 2733 ts_tablet_manager.cc:1397] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260811 22:21:34.672166 2726 raft_consensus.cc:348] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
I20260811 22:21:34.672428 2726 raft_consensus.cc:374] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:34.672487 2726 raft_consensus.cc:729] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53a09e83f756415b9dcb3a71ba6388f9, State: Initialized, Role: FOLLOWER
I20260811 22:21:34.672947 2726 consensus_queue.cc:260] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
I20260811 22:21:34.674005 2726 ts_tablet_manager.cc:1428] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:21:34.675047 2457 heartbeater.cc:503] Master 127.26.125.62:43429 was elected leader, sending a full tablet report...
I20260811 22:21:34.675652 2733 raft_consensus.cc:348] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
I20260811 22:21:34.675885 2733 raft_consensus.cc:374] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:34.675957 2733 raft_consensus.cc:729] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d06862741d2419f833315471a96de25, State: Initialized, Role: FOLLOWER
I20260811 22:21:34.676450 2733 consensus_queue.cc:260] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
I20260811 22:21:34.677172 2695 heartbeater.cc:503] Master 127.26.125.62:43429 was elected leader, sending a full tablet report...
I20260811 22:21:34.677536 2733 ts_tablet_manager.cc:1428] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 22:21:34.843447 2458 tablet.cc:2369] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:21:34.843838 2696 tablet.cc:2369] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:34.855744 2735 raft_consensus.cc:482] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:34.855988 2735 raft_consensus.cc:504] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
I20260811 22:21:34.857180 2735 leader_election.cc:288] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53a09e83f756415b9dcb3a71ba6388f9 (127.26.125.2:35645), 2d06862741d2419f833315471a96de25 (127.26.125.3:43597)
I20260811 22:21:34.861112 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.857481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:35645 (local address 127.26.125.1:45737)
0811 22:21:34.857782 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.857794 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:34.857922 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:34.858177 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:34.858180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:34.858198 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:34.858335 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.858341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.859112 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.859115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.859947 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.859957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.860650 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.860654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.860754 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.860782 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:21:34.860860 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:21:34.860934 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"spinlock_wait_cycles":24192,"thread_start_us":112,"threads_started":1}
I20260811 22:21:34.861120 2716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.857650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:45737 (local address 127.26.125.2:35645)
0811 22:21:34.857781 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.857785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.857888 (+ 103us) server_negotiation.cc:407] Connection header received
0811 22:21:34.858045 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.858047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.858091 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.858155 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:34.858446 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.858994 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.860105 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.860543 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.860640 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.860683 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.860909 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.860989 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:21:34.861388 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.857576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:44873 (local address 127.26.125.3:43597)
0811 22:21:34.857709 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.857714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.858247 (+ 533us) server_negotiation.cc:407] Connection header received
0811 22:21:34.858385 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.858388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.858429 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.858490 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:34.858731 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.859328 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.860359 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.860840 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.860945 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.860993 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.861201 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.861278 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:21:34.861459 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.857481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43597 (local address 127.26.125.1:44873)
0811 22:21:34.858141 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.858153 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:34.858284 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:34.858506 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:34.858509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:34.858524 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:34.858633 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.858638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.859435 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.859438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.860239 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:34.860248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.860963 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.860968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:34.861065 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.861088 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:21:34.861171 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:21:34.861237 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":311,"spinlock_wait_cycles":768,"thread_start_us":120,"threads_started":1}
I20260811 22:21:34.862150 2602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b19ee939bd46d9bf902cfd7c75f418" candidate_uuid: "130e32572fab495c823e3b4efce862e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d06862741d2419f833315471a96de25" is_pre_election: true
I20260811 22:21:34.862155 2379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b19ee939bd46d9bf902cfd7c75f418" candidate_uuid: "130e32572fab495c823e3b4efce862e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53a09e83f756415b9dcb3a71ba6388f9" is_pre_election: true
I20260811 22:21:34.862505 2602 raft_consensus.cc:2454] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130e32572fab495c823e3b4efce862e3 in term 0.
I20260811 22:21:34.862505 2379 raft_consensus.cc:2454] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130e32572fab495c823e3b4efce862e3 in term 0.
I20260811 22:21:34.863065 2037 leader_election.cc:302] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130e32572fab495c823e3b4efce862e3, 53a09e83f756415b9dcb3a71ba6388f9; no voters:
I20260811 22:21:34.863335 2735 raft_consensus.cc:2781] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:34.863420 2735 raft_consensus.cc:482] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:34.863452 2735 raft_consensus.cc:3037] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:34.864535 2735 raft_consensus.cc:504] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
I20260811 22:21:34.864892 2735 leader_election.cc:288] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [CANDIDATE]: Term 1 election: Requested vote from peers 53a09e83f756415b9dcb3a71ba6388f9 (127.26.125.2:35645), 2d06862741d2419f833315471a96de25 (127.26.125.3:43597)
I20260811 22:21:34.865351 2602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b19ee939bd46d9bf902cfd7c75f418" candidate_uuid: "130e32572fab495c823e3b4efce862e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d06862741d2419f833315471a96de25"
I20260811 22:21:34.865408 2379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b19ee939bd46d9bf902cfd7c75f418" candidate_uuid: "130e32572fab495c823e3b4efce862e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53a09e83f756415b9dcb3a71ba6388f9"
I20260811 22:21:34.865494 2602 raft_consensus.cc:3037] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:34.865523 2379 raft_consensus.cc:3037] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:34.866794 2379 raft_consensus.cc:2454] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130e32572fab495c823e3b4efce862e3 in term 1.
I20260811 22:21:34.866794 2602 raft_consensus.cc:2454] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130e32572fab495c823e3b4efce862e3 in term 1.
I20260811 22:21:34.867147 2034 leader_election.cc:302] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130e32572fab495c823e3b4efce862e3, 2d06862741d2419f833315471a96de25; no voters:
I20260811 22:21:34.867323 2735 raft_consensus.cc:2781] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:34.867556 2735 raft_consensus.cc:686] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [term 1 LEADER]: Becoming Leader. State: Replica: 130e32572fab495c823e3b4efce862e3, State: Running, Role: LEADER
I20260811 22:21:34.867882 2735 consensus_queue.cc:237] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } }
W20260811 22:21:34.868345 2236 tablet.cc:2369] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:34.871263 1907 catalog_manager.cc:5495] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130e32572fab495c823e3b4efce862e3 (127.26.125.1). New cstate: current_term: 1 leader_uuid: "130e32572fab495c823e3b4efce862e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130e32572fab495c823e3b4efce862e3" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46879 } health_report { overall_health: HEALTHY } } }
I20260811 22:21:34.888872 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.885287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.26.125.1:46879)
0811 22:21:34.885427 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:34.885432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:34.885447 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:34.885545 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:34.885547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:34.885604 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:34.885701 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:34.885957 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.886525 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.887407 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:34.887627 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:34.887716 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:34.888019 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:34.888470 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:34.888584 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:34.888665 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 22:21:34.888728 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:21:34.895216 2602 raft_consensus.cc:1261] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 1 FOLLOWER]: Refusing update from remote peer 130e32572fab495c823e3b4efce862e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:34.895216 2379 raft_consensus.cc:1261] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 1 FOLLOWER]: Refusing update from remote peer 130e32572fab495c823e3b4efce862e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:34.895960 2735 consensus_queue.cc:1035] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d06862741d2419f833315471a96de25" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:34.896201 2741 consensus_queue.cc:1035] T 19b19ee939bd46d9bf902cfd7c75f418 P 130e32572fab495c823e3b4efce862e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53a09e83f756415b9dcb3a71ba6388f9" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:34.916725 1906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:34.916922 1906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:34.919674 1906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 899c69412c2e4e1c8129946697e3bdf7: Sending DeleteTablet for 3 replicas of tablet 19b19ee939bd46d9bf902cfd7c75f418
I20260811 22:21:34.920627 2563 tablet_service.cc:1514] Processing DeleteTablet for tablet 19b19ee939bd46d9bf902cfd7c75f418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:34 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35202
I20260811 22:21:34.920684 2345 tablet_service.cc:1514] Processing DeleteTablet for tablet 19b19ee939bd46d9bf902cfd7c75f418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:34 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:47724
I20260811 22:21:34.921238 2764 tablet_replica.cc:330] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: stopping tablet replica
I20260811 22:21:34.921222 2122 tablet_service.cc:1514] Processing DeleteTablet for tablet 19b19ee939bd46d9bf902cfd7c75f418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:34 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:48380
I20260811 22:21:34.921373 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 2009
I20260811 22:21:34.921732 2764 raft_consensus.cc:2229] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:34.924453 2764 raft_consensus.cc:2258] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:34.924685 2765 tablet_replica.cc:330] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: stopping tablet replica
I20260811 22:21:34.926924 2765 raft_consensus.cc:2229] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:34.927320 2765 raft_consensus.cc:2258] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:34.928067 2764 ts_tablet_manager.cc:1905] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:34.930716 2764 ts_tablet_manager.cc:1918] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:34.930816 2764 log.cc:1196] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/19b19ee939bd46d9bf902cfd7c75f418
I20260811 22:21:34.931033 2765 ts_tablet_manager.cc:1905] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:34.931169 2764 ts_tablet_manager.cc:1939] T 19b19ee939bd46d9bf902cfd7c75f418 P 2d06862741d2419f833315471a96de25: Deleting consensus metadata
I20260811 22:21:34.932375 1878 catalog_manager.cc:4841] TS 2d06862741d2419f833315471a96de25 (127.26.125.3:43597): tablet 19b19ee939bd46d9bf902cfd7c75f418 (table test-table [id=68b9751ca14244608aae94f73815be73]) successfully deleted
I20260811 22:21:34.933730 2765 ts_tablet_manager.cc:1918] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:34.933847 2765 log.cc:1196] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/19b19ee939bd46d9bf902cfd7c75f418
I20260811 22:21:34.934139 2765 ts_tablet_manager.cc:1939] T 19b19ee939bd46d9bf902cfd7c75f418 P 53a09e83f756415b9dcb3a71ba6388f9: Deleting consensus metadata
I20260811 22:21:34.935385 1881 catalog_manager.cc:4841] TS 53a09e83f756415b9dcb3a71ba6388f9 (127.26.125.2:35645): tablet 19b19ee939bd46d9bf902cfd7c75f418 (table test-table [id=68b9751ca14244608aae94f73815be73]) successfully deleted
W20260811 22:21:34.936434 1881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:46879 (error 108)
I20260811 22:21:34.936645 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 2242
I20260811 22:21:34.940040 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:34.939770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46879 (local address 127.0.0.1:48394)
0811 22:21:34.939894 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:34.939962 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:21:34.940264 1881 catalog_manager.cc:4584] TS 130e32572fab495c823e3b4efce862e3 (127.26.125.1:46879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19b19ee939bd46d9bf902cfd7c75f418: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46879: connect: Connection refused (error 111)
I20260811 22:21:34.952365 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 2468
I20260811 22:21:34.967491 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1850
2026-08-11T22:21:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2677 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T22:21:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:35Z Disabled control of system clock
I20260811 22:21:35.012480 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45185
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:45335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:45185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:21:35.131208 2770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:35.131534 2770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:35.131572 2770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:35.135643 2770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:35.135722 2770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:35.135736 2770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:35.135752 2770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:35.135766 2770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:35.140859 2770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:45185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:35.142026 2770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:35.143460 2770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:35.152163 2789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:35.152163 2784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:35.152266 2770 server_base.cc:1034] running on GCE node
W20260811 22:21:35.152182 2785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:35.153154 2770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:35.154387 2770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:35.155578 2770 hybrid_clock.cc:648] HybridClock initialized: now 1786486895155501 us; error 95 us; skew 500 ppm
I20260811 22:21:35.162242 2770 webserver.cc:460] Webserver started at http://127.26.125.62:35213/ using document root <none> and password file <none>
I20260811 22:21:35.163249 2770 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:35.163363 2770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:35.163640 2770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:35.166148 2770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "7755c901446f4707a30af64db274f922"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "4ec3651de4838a4d54095ee42fff96c2"
server_key_iv: "755c58c300f7ad98120c78f32ee030a4"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.166813 2770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "7755c901446f4707a30af64db274f922"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "4ec3651de4838a4d54095ee42fff96c2"
server_key_iv: "755c58c300f7ad98120c78f32ee030a4"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.170915 2770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:21:35.173430 2795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.174768 2770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 22:21:35.174958 2770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "7755c901446f4707a30af64db274f922"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "4ec3651de4838a4d54095ee42fff96c2"
server_key_iv: "755c58c300f7ad98120c78f32ee030a4"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.175076 2770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:35.179883 2770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:35.180552 2770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:35.180725 2770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:35.188100 2770 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:45185
I20260811 22:21:35.188119 2898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:45185 every 8 connection(s)
I20260811 22:21:35.189136 2770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:35.192687 2899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:35.198535 2899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922: Bootstrap starting.
I20260811 22:21:35.199417 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 2770
I20260811 22:21:35.199546 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:35.199846 27124 external_mini_cluster.cc:1428] Setting key 64e94f37cea9a0677e2374ce05d5bce8
I20260811 22:21:35.201777 2899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:35.202559 2899 log.cc:824] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:35.204800 2899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922: No bootstrap required, opened a new log
I20260811 22:21:35.205845 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.201199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.26.125.62:45185)
0811 22:21:35.201951 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:35.201969 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:35.202024 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:21:35.202752 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:35.202773 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:35.203273 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:35.203716 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:35.204527 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:35.204532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:35.204544 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:35.204555 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:35.204615 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:35.204631 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:35.204734 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:35.204858 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 22:21:35.205113 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":156,"threads_started":1}
I20260811 22:21:35.208525 2899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7755c901446f4707a30af64db274f922" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45185 } }
I20260811 22:21:35.208767 2899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:35.208827 2899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7755c901446f4707a30af64db274f922, State: Initialized, Role: FOLLOWER
I20260811 22:21:35.209260 2899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7755c901446f4707a30af64db274f922" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45185 } }
I20260811 22:21:35.209398 2899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:35.209462 2899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:35.209565 2899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:35.210649 2899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7755c901446f4707a30af64db274f922" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45185 } }
I20260811 22:21:35.211045 2899 leader_election.cc:302] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7755c901446f4707a30af64db274f922; no voters:
I20260811 22:21:35.211411 2899 leader_election.cc:288] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:35.211515 2908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:35.211733 2908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [term 1 LEADER]: Becoming Leader. State: Replica: 7755c901446f4707a30af64db274f922, State: Running, Role: LEADER
I20260811 22:21:35.212078 2908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7755c901446f4707a30af64db274f922" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45185 } }
I20260811 22:21:35.212703 2899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:35.213768 2909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7755c901446f4707a30af64db274f922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7755c901446f4707a30af64db274f922" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45185 } } }
I20260811 22:21:35.214112 2909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:35.213869 2911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7755c901446f4707a30af64db274f922. Latest consensus state: current_term: 1 leader_uuid: "7755c901446f4707a30af64db274f922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7755c901446f4707a30af64db274f922" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45185 } } }
I20260811 22:21:35.214250 2911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:35.214566 2921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:35.217581 2921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:35.223673 2921 catalog_manager.cc:1347] Generated new cluster ID: 2b031662f41c4b45b62d60a1ea2f444f
I20260811 22:21:35.223765 2921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:35.265504 2921 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:35.265929 2921 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:35.278985 2921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922: Generated new TSK 0
I20260811 22:21:35.279847 2921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:21:35.283013 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45185
--builtin_ntp_servers=127.26.125.20:45335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:21:35.402258 2937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:35.402565 2937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:35.402604 2937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:35.406709 2937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:35.406780 2937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:35.406836 2937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:35.411799 2937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:35.413133 2937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:35.414760 2937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:35.425374 2937 server_base.cc:1034] running on GCE node
W20260811 22:21:35.425320 2948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:35.425320 2944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:35.425323 2946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:35.426046 2937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:35.426762 2937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:35.427925 2937 hybrid_clock.cc:648] HybridClock initialized: now 1786486895427893 us; error 46 us; skew 500 ppm
I20260811 22:21:35.429819 2937 webserver.cc:460] Webserver started at http://127.26.125.1:40871/ using document root <none> and password file <none>
I20260811 22:21:35.430569 2937 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:35.430644 2937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:35.430891 2937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:35.433336 2937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "fc25545d58454d9d83d536e976aaefa8"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "2b7f024cbaa486e814f67be0ed38f3ac"
server_key_iv: "ae60861085b954d6b1850882f8beb68f"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.433939 2937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "fc25545d58454d9d83d536e976aaefa8"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "2b7f024cbaa486e814f67be0ed38f3ac"
server_key_iv: "ae60861085b954d6b1850882f8beb68f"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.438068 2937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:21:35.441135 2961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.442597 2937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:35.442724 2937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "fc25545d58454d9d83d536e976aaefa8"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "2b7f024cbaa486e814f67be0ed38f3ac"
server_key_iv: "ae60861085b954d6b1850882f8beb68f"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.442842 2937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:35.447672 2937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:35.448318 2937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:35.448485 2937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:35.449064 2937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:35.450008 2937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:35.450058 2937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.450135 2937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:35.450160 2937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.459139 2937 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:36861
I20260811 22:21:35.459161 3158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:36861 every 8 connection(s)
I20260811 22:21:35.460116 2937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:35.460930 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 2937
I20260811 22:21:35.461050 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:35.461369 27124 external_mini_cluster.cc:1428] Setting key 01552866908eacc23edc51cac712d986
I20260811 22:21:35.463357 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45185
--builtin_ntp_servers=127.26.125.20:45335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:21:35.469224 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.462009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:58091 (local address 127.26.125.62:45185)
0811 22:21:35.462148 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:35.462152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:35.465572 (+ 3420us) server_negotiation.cc:407] Connection header received
0811 22:21:35.467262 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:35.467266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:35.467313 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:35.467389 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:35.468760 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:35.468763 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:35.468765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:35.468777 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:35.468799 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:35.468807 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:35.468893 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:35.469030 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 22:21:35.469079 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:21:35.470232 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.462495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45185 (local address 127.26.125.1:58091)
0811 22:21:35.465415 (+ 2920us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:35.465462 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:35.467024 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:35.467619 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:35.467627 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:35.467729 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:35.468365 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:21:35.468366 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:35.468410 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:21:35.468517 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:35.468884 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:35.468893 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:35.468900 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:21:35.469074 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 22:21:35.469415 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2478,"thread_start_us":133,"threads_started":1}
I20260811 22:21:35.471581 3159 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45185
I20260811 22:21:35.471866 3159 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:35.472414 3159 heartbeater.cc:511] Master 127.26.125.62:45185 requested a full tablet report, sending...
I20260811 22:21:35.474499 2824 ts_manager.cc:194] Registered new tserver with Master: fc25545d58454d9d83d536e976aaefa8 (127.26.125.1:36861)
W20260811 22:21:35.587483 3163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:35.587777 3163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:35.587821 3163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:35.591873 3163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:35.591940 3163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:35.592000 3163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:35.596876 3163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:35.597981 3163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:35.599444 3163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:35.609424 3173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:35.609530 3176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:35.609586 3163 server_base.cc:1034] running on GCE node
W20260811 22:21:35.609764 3178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:35.610216 3163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:35.610945 3163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:35.612115 3163 hybrid_clock.cc:648] HybridClock initialized: now 1786486895612090 us; error 39 us; skew 500 ppm
I20260811 22:21:35.614231 3163 webserver.cc:460] Webserver started at http://127.26.125.2:35211/ using document root <none> and password file <none>
I20260811 22:21:35.614960 3163 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:35.615041 3163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:35.615262 3163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:35.617488 3163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "c71801f035bc439d908cb710dda0bf60"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "2ca8b62320ad17044969b6f709099d8a"
server_key_iv: "0eaacff14e0215b6dc89ff952e183e5f"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.618088 3163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "c71801f035bc439d908cb710dda0bf60"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "2ca8b62320ad17044969b6f709099d8a"
server_key_iv: "0eaacff14e0215b6dc89ff952e183e5f"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.622202 3163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:35.625013 3191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.626508 3163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:35.626641 3163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "c71801f035bc439d908cb710dda0bf60"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "2ca8b62320ad17044969b6f709099d8a"
server_key_iv: "0eaacff14e0215b6dc89ff952e183e5f"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.626818 3163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:35.632009 3163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:35.632663 3163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:35.632827 3163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:35.633399 3163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:35.634364 3163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:35.634415 3163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.634460 3163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:35.634490 3163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.643288 3163 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:46171
I20260811 22:21:35.643313 3386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:46171 every 8 connection(s)
I20260811 22:21:35.644246 3163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:35.650842 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.646199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:45823 (local address 127.26.125.62:45185)
0811 22:21:35.646325 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:35.646329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:35.647627 (+ 1298us) server_negotiation.cc:407] Connection header received
0811 22:21:35.648976 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:35.648980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:35.649038 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:35.649112 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:35.650338 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:35.650341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:35.650343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:35.650356 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:35.650379 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:35.650387 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:35.650470 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:35.650640 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:21:35.650691 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:21:35.651942 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 3163
I20260811 22:21:35.651940 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.646627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45185 (local address 127.26.125.2:45823)
0811 22:21:35.647436 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:35.647482 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:35.648735 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:35.649319 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:35.649326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:35.649427 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:35.650044 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:21:35.650046 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:35.650101 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:21:35.650176 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:35.650460 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:35.650469 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:35.650475 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:21:35.650677 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 22:21:35.650976 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":153,"threads_started":1}
I20260811 22:21:35.652078 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:35.652402 27124 external_mini_cluster.cc:1428] Setting key 06829c090a873d2e63439cdd2323b7a0
I20260811 22:21:35.653610 3388 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45185
I20260811 22:21:35.653888 3388 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:35.654364 3388 heartbeater.cc:511] Master 127.26.125.62:45185 requested a full tablet report, sending...
I20260811 22:21:35.654516 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45185
--builtin_ntp_servers=127.26.125.20:45335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:21:35.655606 2824 ts_manager.cc:194] Registered new tserver with Master: c71801f035bc439d908cb710dda0bf60 (127.26.125.2:46171)
W20260811 22:21:35.773940 3393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:35.774238 3393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:35.774296 3393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:35.778625 3393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:35.778713 3393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:35.778793 3393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:35.784058 3393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:35.785336 3393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:35.786804 3393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:35.796900 3399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:35.796900 3400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:35.796954 3393 server_base.cc:1034] running on GCE node
W20260811 22:21:35.797165 3402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:35.797626 3393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:35.798264 3393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:35.799429 3393 hybrid_clock.cc:648] HybridClock initialized: now 1786486895799392 us; error 47 us; skew 500 ppm
I20260811 22:21:35.801314 3393 webserver.cc:460] Webserver started at http://127.26.125.3:33103/ using document root <none> and password file <none>
I20260811 22:21:35.801936 3393 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:35.802000 3393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:35.802178 3393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:35.804369 3393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "74759d979c804646a3a363073d2beae7"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "b5a14a60f21dbe57bd2662027f2b56d2"
server_key_iv: "d9f144fb9644670633de8b48d7a4f601"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.804916 3393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "74759d979c804646a3a363073d2beae7"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "b5a14a60f21dbe57bd2662027f2b56d2"
server_key_iv: "d9f144fb9644670633de8b48d7a4f601"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.809106 3393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:35.812088 3415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.813675 3393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:21:35.813818 3393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "74759d979c804646a3a363073d2beae7"
format_stamp: "Formatted at 2026-08-11 22:21:35 on dist-test-slave-5bn2"
server_key: "b5a14a60f21dbe57bd2662027f2b56d2"
server_key_iv: "d9f144fb9644670633de8b48d7a4f601"
server_key_version: "encryptionkey@0"
I20260811 22:21:35.813941 3393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:35.818696 3393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:35.819376 3393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:35.819530 3393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:35.820060 3393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:35.821036 3393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:35.821089 3393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.821130 3393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:35.821149 3393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:35.829862 3393 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:42297
I20260811 22:21:35.829881 3618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:42297 every 8 connection(s)
I20260811 22:21:35.831029 3393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:35.832347 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 3393
I20260811 22:21:35.832472 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:35.832775 27124 external_mini_cluster.cc:1428] Setting key 9f8b604ad837947d970c482855017cf8
I20260811 22:21:35.837813 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.833178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:36865 (local address 127.26.125.62:45185)
0811 22:21:35.833311 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:35.833315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:35.834750 (+ 1435us) server_negotiation.cc:407] Connection header received
0811 22:21:35.836173 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:35.836176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:35.836215 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:35.836280 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:35.837397 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:35.837401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:35.837403 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:35.837415 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:35.837437 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:35.837445 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:35.837520 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:35.837654 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 22:21:35.837701 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:21:35.838701 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.833716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45185 (local address 127.26.125.3:36865)
0811 22:21:35.834580 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:35.834638 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:35.835961 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:35.836434 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:35.836442 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:35.836540 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:35.837123 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:21:35.837124 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:35.837170 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:21:35.837245 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:35.837509 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:35.837514 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:35.837517 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:21:35.837644 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 22:21:35.837851 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":153,"threads_started":1}
I20260811 22:21:35.839859 3620 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45185
I20260811 22:21:35.840090 3620 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:35.840529 3620 heartbeater.cc:511] Master 127.26.125.62:45185 requested a full tablet report, sending...
I20260811 22:21:35.841542 2824 ts_manager.cc:194] Registered new tserver with Master: 74759d979c804646a3a363073d2beae7 (127.26.125.3:42297)
I20260811 22:21:35.847023 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:21:35.850620 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.848936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.26.125.62:45185)
0811 22:21:35.849073 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:35.849078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:35.849241 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:21:35.849363 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:35.849366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:35.849405 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:35.849480 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:35.850215 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:35.850219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:35.850222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:35.850234 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:35.850257 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:35.850264 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:35.850396 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:35.850434 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 22:21:35.850473 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:21:35.853978 2824 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:35.857074 2824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:35.869387 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.865867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46171 (local address 127.0.0.1:35412)
0811 22:21:35.867114 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:35.867167 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:35.867287 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:35.867804 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:35.867810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:35.867904 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:35.868514 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:21:35.868516 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:35.868538 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:21:35.868611 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:35.869041 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:35.869051 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:35.869057 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:21:35.869163 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:21:35.869234 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":95,"threads_started":1}
I20260811 22:21:35.869774 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.866873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50354 (local address 127.26.125.3:42297)
0811 22:21:35.867246 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:35.867265 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:35.867708 (+ 443us) server_negotiation.cc:407] Connection header received
0811 22:21:35.867767 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:35.867773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:35.867968 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:35.868107 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:35.869176 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:35.869182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:35.869194 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:35.869204 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:35.869250 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:35.869287 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:35.869380 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:35.869552 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 22:21:35.869632 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":155,"threads_started":1}
I20260811 22:21:35.869787 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.865954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:36861 (local address 127.0.0.1:56484)
0811 22:21:35.867397 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:35.867410 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:35.867490 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:35.868438 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:35.868442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:35.868456 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:35.868893 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:21:35.868895 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:35.868905 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:21:35.868930 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:35.869252 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:35.869255 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:35.869258 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:21:35.869338 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:21:35.869374 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1387,"thread_start_us":93,"threads_started":1}
I20260811 22:21:35.869812 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.865990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.26.125.2:46171)
0811 22:21:35.866810 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:35.866821 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:35.867272 (+ 451us) server_negotiation.cc:407] Connection header received
0811 22:21:35.867380 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:35.867386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:35.867601 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:35.867737 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:35.868764 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:35.868770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:35.868784 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:35.868798 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:35.868842 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:35.868882 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:35.868999 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:35.869586 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 22:21:35.869663 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":131,"threads_started":1}
I20260811 22:21:35.870177 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.866665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:42297 (local address 127.0.0.1:50354)
0811 22:21:35.867616 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:35.867625 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:35.867683 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:35.868657 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:35.868660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:35.868673 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:35.869018 (+ 345us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:21:35.869019 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:35.869035 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:21:35.869058 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:35.869419 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:35.869421 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:35.869425 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:21:35.870020 (+ 595us) client_negotiation.cc:240] Negotiation successful
0811 22:21:35.870056 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"thread_start_us":99,"threads_started":1}
I20260811 22:21:35.870579 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:35.866196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.26.125.1:36861)
0811 22:21:35.866597 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:35.866610 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:35.867506 (+ 896us) server_negotiation.cc:407] Connection header received
0811 22:21:35.867562 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:35.867567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:35.867727 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:35.867840 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:35.869059 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:35.869065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:35.869077 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:35.869090 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:35.869136 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:35.869175 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:35.869270 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:35.870351 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0811 22:21:35.870431 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":91,"threads_started":1}
I20260811 22:21:35.873145 3496 tablet_service.cc:1467] Processing CreateTablet for tablet 983c9f8e4933463cbfdb19e3c184da55 (DEFAULT_TABLE table=test-table [id=96e99db8b3294236a3af0293ef35feee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:35.873142 3271 tablet_service.cc:1467] Processing CreateTablet for tablet 983c9f8e4933463cbfdb19e3c184da55 (DEFAULT_TABLE table=test-table [id=96e99db8b3294236a3af0293ef35feee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:35.873277 3045 tablet_service.cc:1467] Processing CreateTablet for tablet 983c9f8e4933463cbfdb19e3c184da55 (DEFAULT_TABLE table=test-table [id=96e99db8b3294236a3af0293ef35feee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:35.874225 3496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 983c9f8e4933463cbfdb19e3c184da55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:35.874241 3271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 983c9f8e4933463cbfdb19e3c184da55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:35.874241 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 983c9f8e4933463cbfdb19e3c184da55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:35.887753 3650 tablet_bootstrap.cc:492] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Bootstrap starting.
I20260811 22:21:35.890585 3650 tablet_bootstrap.cc:654] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:35.890964 3651 tablet_bootstrap.cc:492] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: Bootstrap starting.
I20260811 22:21:35.891631 3650 log.cc:824] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:35.892279 3649 tablet_bootstrap.cc:492] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8: Bootstrap starting.
I20260811 22:21:35.893498 3651 tablet_bootstrap.cc:654] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:35.894079 3650 tablet_bootstrap.cc:492] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: No bootstrap required, opened a new log
I20260811 22:21:35.894308 3650 ts_tablet_manager.cc:1397] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 22:21:35.894426 3651 log.cc:824] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:35.894666 3649 tablet_bootstrap.cc:654] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:35.895726 3649 log.cc:824] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:35.897276 3651 tablet_bootstrap.cc:492] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: No bootstrap required, opened a new log
I20260811 22:21:35.897482 3651 ts_tablet_manager.cc:1397] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:21:35.897910 3649 tablet_bootstrap.cc:492] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8: No bootstrap required, opened a new log
I20260811 22:21:35.898115 3649 ts_tablet_manager.cc:1397] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:21:35.899180 3650 raft_consensus.cc:348] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:35.899494 3650 raft_consensus.cc:374] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:35.899550 3650 raft_consensus.cc:729] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74759d979c804646a3a363073d2beae7, State: Initialized, Role: FOLLOWER
I20260811 22:21:35.900039 3650 consensus_queue.cc:260] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:35.900820 3620 heartbeater.cc:503] Master 127.26.125.62:45185 was elected leader, sending a full tablet report...
I20260811 22:21:35.901221 3650 ts_tablet_manager.cc:1428] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:21:35.902498 3649 raft_consensus.cc:348] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:35.902724 3649 raft_consensus.cc:374] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:35.902526 3651 raft_consensus.cc:348] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:35.902777 3649 raft_consensus.cc:729] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc25545d58454d9d83d536e976aaefa8, State: Initialized, Role: FOLLOWER
I20260811 22:21:35.902791 3651 raft_consensus.cc:374] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:35.902838 3651 raft_consensus.cc:729] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c71801f035bc439d908cb710dda0bf60, State: Initialized, Role: FOLLOWER
I20260811 22:21:35.903319 3649 consensus_queue.cc:260] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:35.903376 3651 consensus_queue.cc:260] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:35.903985 3159 heartbeater.cc:503] Master 127.26.125.62:45185 was elected leader, sending a full tablet report...
I20260811 22:21:35.904312 3388 heartbeater.cc:503] Master 127.26.125.62:45185 was elected leader, sending a full tablet report...
I20260811 22:21:35.904438 3651 ts_tablet_manager.cc:1428] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260811 22:21:35.905470 3649 ts_tablet_manager.cc:1428] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 22:21:35.961283 3160 tablet.cc:2369] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:36.052111 3658 raft_consensus.cc:482] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:36.052358 3658 raft_consensus.cc:504] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:36.053690 3658 leader_election.cc:288] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc25545d58454d9d83d536e976aaefa8 (127.26.125.1:36861), 74759d979c804646a3a363073d2beae7 (127.26.125.3:42297)
I20260811 22:21:36.055614 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.053988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:36861 (local address 127.26.125.2:40807)
0811 22:21:36.054141 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:36.054156 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:36.054246 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:36.054532 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:36.054536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:36.054553 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:36.055095 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:21:36.055096 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:36.055107 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:21:36.055135 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:36.055373 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:36.055376 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:36.055379 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:21:36.055448 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:21:36.055482 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 22:21:36.055614 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.054085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:40807 (local address 127.26.125.1:36861)
0811 22:21:36.054206 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:36.054210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:36.054228 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:21:36.054347 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:36.054350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:36.054395 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:36.054488 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:36.055262 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:36.055265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:36.055267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:36.055278 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:36.055297 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:36.055305 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:36.055382 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:36.055445 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 22:21:36.055485 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:21:36.056082 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.054279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:42297 (local address 127.26.125.2:58255)
0811 22:21:36.054700 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:36.054713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:36.054800 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:36.055067 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:36.055070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:36.055083 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:36.055520 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:21:36.055522 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:36.055533 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:21:36.055559 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:36.055826 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:36.055829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:36.055833 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:21:36.055909 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:21:36.055940 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":169,"threads_started":1}
I20260811 22:21:36.056098 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.054431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:58255 (local address 127.26.125.3:42297)
0811 22:21:36.054559 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:36.054563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:36.054780 (+ 217us) server_negotiation.cc:407] Connection header received
0811 22:21:36.054930 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:36.054933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:36.054970 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:36.055031 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:36.055671 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:36.055674 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:36.055676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:36.055688 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:36.055714 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:36.055725 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:36.055804 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:36.055916 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 22:21:36.055955 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:21:36.056360 3077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "983c9f8e4933463cbfdb19e3c184da55" candidate_uuid: "c71801f035bc439d908cb710dda0bf60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc25545d58454d9d83d536e976aaefa8" is_pre_election: true
I20260811 22:21:36.056700 3077 raft_consensus.cc:2454] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c71801f035bc439d908cb710dda0bf60 in term 0.
I20260811 22:21:36.056746 3534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "983c9f8e4933463cbfdb19e3c184da55" candidate_uuid: "c71801f035bc439d908cb710dda0bf60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74759d979c804646a3a363073d2beae7" is_pre_election: true
I20260811 22:21:36.057049 3534 raft_consensus.cc:2454] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c71801f035bc439d908cb710dda0bf60 in term 0.
I20260811 22:21:36.057153 3199 leader_election.cc:302] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c71801f035bc439d908cb710dda0bf60, fc25545d58454d9d83d536e976aaefa8; no voters:
I20260811 22:21:36.057440 3658 raft_consensus.cc:2781] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:36.057518 3658 raft_consensus.cc:482] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:36.057577 3658 raft_consensus.cc:3037] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:36.058615 3658 raft_consensus.cc:504] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:36.058954 3658 leader_election.cc:288] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [CANDIDATE]: Term 1 election: Requested vote from peers fc25545d58454d9d83d536e976aaefa8 (127.26.125.1:36861), 74759d979c804646a3a363073d2beae7 (127.26.125.3:42297)
I20260811 22:21:36.059309 3077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "983c9f8e4933463cbfdb19e3c184da55" candidate_uuid: "c71801f035bc439d908cb710dda0bf60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc25545d58454d9d83d536e976aaefa8"
I20260811 22:21:36.059310 3534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "983c9f8e4933463cbfdb19e3c184da55" candidate_uuid: "c71801f035bc439d908cb710dda0bf60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74759d979c804646a3a363073d2beae7"
I20260811 22:21:36.059423 3077 raft_consensus.cc:3037] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:36.059427 3534 raft_consensus.cc:3037] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:36.060631 3077 raft_consensus.cc:2454] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c71801f035bc439d908cb710dda0bf60 in term 1.
I20260811 22:21:36.060693 3534 raft_consensus.cc:2454] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c71801f035bc439d908cb710dda0bf60 in term 1.
I20260811 22:21:36.060952 3199 leader_election.cc:302] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74759d979c804646a3a363073d2beae7, c71801f035bc439d908cb710dda0bf60; no voters:
I20260811 22:21:36.061123 3658 raft_consensus.cc:2781] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:36.061333 3658 raft_consensus.cc:686] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 1 LEADER]: Becoming Leader. State: Replica: c71801f035bc439d908cb710dda0bf60, State: Running, Role: LEADER
I20260811 22:21:36.061690 3658 consensus_queue.cc:237] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } }
I20260811 22:21:36.064918 2823 catalog_manager.cc:5495] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 reported cstate change: term changed from 0 to 1, leader changed from <none> to c71801f035bc439d908cb710dda0bf60 (127.26.125.2). New cstate: current_term: 1 leader_uuid: "c71801f035bc439d908cb710dda0bf60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c71801f035bc439d908cb710dda0bf60" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 } health_report { overall_health: UNKNOWN } } }
W20260811 22:21:36.082108 3623 tablet.cc:2369] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:36.113368 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.111903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35418 (local address 127.26.125.2:46171)
0811 22:21:36.112031 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:36.112034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:36.112048 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:36.112156 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:36.112159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:36.112225 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:36.112294 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:36.112892 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:36.112895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:36.112897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:36.112908 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:36.112944 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:36.112953 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:36.113026 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:36.113170 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 22:21:36.113227 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:36.120048 3534 raft_consensus.cc:1261] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 1 FOLLOWER]: Refusing update from remote peer c71801f035bc439d908cb710dda0bf60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:36.120060 3077 raft_consensus.cc:1261] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [term 1 FOLLOWER]: Refusing update from remote peer c71801f035bc439d908cb710dda0bf60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:36.120790 3658 consensus_queue.cc:1035] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74759d979c804646a3a363073d2beae7" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:36.120990 3661 consensus_queue.cc:1035] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc25545d58454d9d83d536e976aaefa8" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 36861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:36.128660 3669 mvcc.cc:205] Tried to move back new op lower bound from 7317450326499430400 to 7317450326271221760. Current Snapshot: MvccSnapshot[applied={T|T < 7317450326499430400}]
I20260811 22:21:36.133026 3672 mvcc.cc:205] Tried to move back new op lower bound from 7317450326499430400 to 7317450326271221760. Current Snapshot: MvccSnapshot[applied={T|T < 7317450326499430400}]
I20260811 22:21:36.138072 2823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:36.138242 2823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:36.140825 2823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7755c901446f4707a30af64db274f922: Sending DeleteTablet for 3 replicas of tablet 983c9f8e4933463cbfdb19e3c184da55
I20260811 22:21:36.141709 3271 tablet_service.cc:1514] Processing DeleteTablet for tablet 983c9f8e4933463cbfdb19e3c184da55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:36 UTC) from {username='slave'} at 127.0.0.1:35412
I20260811 22:21:36.141690 3045 tablet_service.cc:1514] Processing DeleteTablet for tablet 983c9f8e4933463cbfdb19e3c184da55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:36 UTC) from {username='slave'} at 127.0.0.1:56484
I20260811 22:21:36.142259 3496 tablet_service.cc:1514] Processing DeleteTablet for tablet 983c9f8e4933463cbfdb19e3c184da55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:36 UTC) from {username='slave'} at 127.0.0.1:50354
I20260811 22:21:36.142449 3689 tablet_replica.cc:330] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8: stopping tablet replica
I20260811 22:21:36.142791 3691 tablet_replica.cc:330] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: stopping tablet replica
I20260811 22:21:36.142800 3689 raft_consensus.cc:2229] T 983c9f8e4933463cbfdb19e3c184da55 P fc25545d58454d9d83d536e976aaefa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:36.143122 3691 raft_consensus.cc:2229] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:36.143179 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 2937
I20260811 22:21:36.155848 3688 tablet_replica.cc:330] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: stopping tablet replica
I20260811 22:21:36.156172 3688 raft_consensus.cc:2229] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:36.156503 3688 raft_consensus.cc:2258] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:36.156898 3691 raft_consensus.cc:2258] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:36.158661 3691 ts_tablet_manager.cc:1905] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:36.158661 3688 ts_tablet_manager.cc:1905] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:36.161585 3691 ts_tablet_manager.cc:1918] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:36.161585 3688 ts_tablet_manager.cc:1918] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:36.161711 3691 log.cc:1196] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/983c9f8e4933463cbfdb19e3c184da55
I20260811 22:21:36.161711 3688 log.cc:1196] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/983c9f8e4933463cbfdb19e3c184da55
I20260811 22:21:36.162041 3688 ts_tablet_manager.cc:1939] T 983c9f8e4933463cbfdb19e3c184da55 P c71801f035bc439d908cb710dda0bf60: Deleting consensus metadata
I20260811 22:21:36.162041 3691 ts_tablet_manager.cc:1939] T 983c9f8e4933463cbfdb19e3c184da55 P 74759d979c804646a3a363073d2beae7: Deleting consensus metadata
I20260811 22:21:36.163439 2802 catalog_manager.cc:4841] TS 74759d979c804646a3a363073d2beae7 (127.26.125.3:42297): tablet 983c9f8e4933463cbfdb19e3c184da55 (table test-table [id=96e99db8b3294236a3af0293ef35feee]) successfully deleted
I20260811 22:21:36.163460 2799 catalog_manager.cc:4841] TS c71801f035bc439d908cb710dda0bf60 (127.26.125.2:46171): tablet 983c9f8e4933463cbfdb19e3c184da55 (table test-table [id=96e99db8b3294236a3af0293ef35feee]) successfully deleted
W20260811 22:21:36.165704 2802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:36861 (error 108)
I20260811 22:21:36.165931 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 3163
I20260811 22:21:36.206737 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.206369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:36861 (local address 127.0.0.1:56492)
0811 22:21:36.206535 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:36.206633 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:36861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:21:36.207005 2802 catalog_manager.cc:4584] TS fc25545d58454d9d83d536e976aaefa8 (127.26.125.1:36861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 983c9f8e4933463cbfdb19e3c184da55: Network error: Client connection negotiation failed: client connection to 127.26.125.1:36861: connect: Connection refused (error 111)
I20260811 22:21:36.211782 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 3393
I20260811 22:21:36.231793 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 2770
2026-08-11T22:21:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1260 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 22:21:36.263145 27124 mini_oidc.cc:152] Starting JWKS server
I20260811 22:21:36.265151 27124 webserver.cc:460] Webserver started at https://127.0.0.1:34219/ using document root <none> and password file <none>
I20260811 22:21:36.265329 27124 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:21:36.265625 27124 webserver.cc:460] Webserver started at http://127.0.0.1:35551/ using document root <none> and password file <none>
2026-08-11T22:21:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:36Z Disabled control of system clock
I20260811 22:21:36.276803 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:34251
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:45821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:34251
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34219/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/testchainca.cert with env {}
W20260811 22:21:36.392305 3704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:36.392607 3704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:36.392666 3704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:36.396736 3704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:36.396807 3704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:36.396827 3704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:36.396847 3704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:36.396867 3704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:21:36.396908 3704 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:21:36.396935 3704 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34219/jwks.json
I20260811 22:21:36.402499 3704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:34251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:34251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34219/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:36.403656 3704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:36.404724 3704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:36.413545 3704 server_base.cc:1034] running on GCE node
W20260811 22:21:36.413558 3711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:36.413518 3715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:36.413514 3718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:36.414485 3704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:36.415757 3704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:36.416919 3704 hybrid_clock.cc:648] HybridClock initialized: now 1786486896416903 us; error 31 us; skew 500 ppm
I20260811 22:21:36.418828 3704 webserver.cc:460] Webserver started at http://127.26.125.62:34669/ using document root <none> and password file <none>
I20260811 22:21:36.419481 3704 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:36.419576 3704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:36.419796 3704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:36.422014 3704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "eb45fc1c255b4a2193952d892144f79e"
format_stamp: "Formatted at 2026-08-11 22:21:36 on dist-test-slave-5bn2"
server_key: "684e7289df8bd13166443707afd6cf70"
server_key_iv: "d627942ed2a64b545bf2c33801c4796b"
server_key_version: "encryptionkey@0"
I20260811 22:21:36.422585 3704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "eb45fc1c255b4a2193952d892144f79e"
format_stamp: "Formatted at 2026-08-11 22:21:36 on dist-test-slave-5bn2"
server_key: "684e7289df8bd13166443707afd6cf70"
server_key_iv: "d627942ed2a64b545bf2c33801c4796b"
server_key_version: "encryptionkey@0"
I20260811 22:21:36.426524 3704 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 22:21:36.429239 3730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:36.430559 3704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:36.430681 3704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "eb45fc1c255b4a2193952d892144f79e"
format_stamp: "Formatted at 2026-08-11 22:21:36 on dist-test-slave-5bn2"
server_key: "684e7289df8bd13166443707afd6cf70"
server_key_iv: "d627942ed2a64b545bf2c33801c4796b"
server_key_version: "encryptionkey@0"
I20260811 22:21:36.430785 3704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:36.447295 3704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:36.448046 3704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:36.448235 3704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:36.455396 3704 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:34251
I20260811 22:21:36.455461 3812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:34251 every 8 connection(s)
I20260811 22:21:36.456419 3704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:36.459790 3813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:36.463748 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 3704
I20260811 22:21:36.463960 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:36.464253 27124 external_mini_cluster.cc:1428] Setting key 426458a3f5a1fb1b4c6e1d2d85fce55a
I20260811 22:21:36.465992 3813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e: Bootstrap starting.
I20260811 22:21:36.469017 3813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:36.469965 3813 log.cc:824] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:36.472402 3813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e: No bootstrap required, opened a new log
I20260811 22:21:36.475313 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.465735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35426 (local address 127.26.125.62:34251)
0811 22:21:36.466618 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:36.466639 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:36.466685 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:21:36.467603 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:36.467631 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:36.468180 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:36.468567 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:36.469067 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.470521 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.471522 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.472212 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.472373 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:36.473173 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:36.473179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:36.473195 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:36.473697 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:36.473773 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:36.473801 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:36.473910 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:36.474022 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 22:21:36.474306 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":144,"threads_started":1}
I20260811 22:21:36.476872 3813 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb45fc1c255b4a2193952d892144f79e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34251 } }
I20260811 22:21:36.477151 3813 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:36.477211 3813 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45fc1c255b4a2193952d892144f79e, State: Initialized, Role: FOLLOWER
I20260811 22:21:36.477679 3813 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb45fc1c255b4a2193952d892144f79e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34251 } }
I20260811 22:21:36.477825 3813 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:36.477895 3813 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:36.477996 3813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:36.478422 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:21:36.479178 3813 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb45fc1c255b4a2193952d892144f79e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34251 } }
I20260811 22:21:36.479563 3813 leader_election.cc:302] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb45fc1c255b4a2193952d892144f79e; no voters:
I20260811 22:21:36.479895 3813 leader_election.cc:288] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:36.480014 3826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:36.480228 3826 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [term 1 LEADER]: Becoming Leader. State: Replica: eb45fc1c255b4a2193952d892144f79e, State: Running, Role: LEADER
I20260811 22:21:36.480561 3826 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb45fc1c255b4a2193952d892144f79e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34251 } }
I20260811 22:21:36.481138 3813 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:36.482192 3827 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb45fc1c255b4a2193952d892144f79e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb45fc1c255b4a2193952d892144f79e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34251 } } }
I20260811 22:21:36.482321 3827 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:36.482728 3835 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb45fc1c255b4a2193952d892144f79e. Latest consensus state: current_term: 1 leader_uuid: "eb45fc1c255b4a2193952d892144f79e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb45fc1c255b4a2193952d892144f79e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34251 } } }
I20260811 22:21:36.482920 3835 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:36.482951 3849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:36.485859 3849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:36.491585 3849 catalog_manager.cc:1347] Generated new cluster ID: 04df61d58d2b428b9fd7d4debdce8ae3
I20260811 22:21:36.491676 3849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:36.505606 3849 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:36.506630 3849 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:36.521404 3849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb45fc1c255b4a2193952d892144f79e: Generated new TSK 0
I20260811 22:21:36.522219 3849 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:21:36.559816 3877 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 22:21:36.560009 3877 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:21:36.560724 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.535718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.26.125.62:34251)
0811 22:21:36.535853 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:36.535858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:36.536070 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:21:36.536297 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:36.536300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:36.536363 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:36.536485 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:21:36.536825 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.537669 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.538554 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.538951 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.539052 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:36.539394 (+ 342us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:21:36.560252 (+ 20858us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:21:36.560571 (+ 319us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45364: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 22:21:36.560818 3821 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45364: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 22:21:36.562422 3876 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:36.535602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:34251 (local address 127.0.0.1:45364)
0811 22:21:36.535985 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:36.536000 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:36.536136 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:36.536582 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:36.536585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:36.536593 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:21:36.536716 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:36.536722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.537811 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.537814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:36.538447 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:36.538455 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.539058 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.539061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:36.539136 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:36.539194 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:21:36.560557 (+ 21363us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:21:36.560792 (+ 235us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.125.62:34251: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.26.125.62:34251: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 22:21:36.563298 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 3704
2026-08-11T22:21:36Z chronyd exiting
I20260811 22:21:36.613106 27124 test_util.cc:175] -----------------------------------------------
I20260811 22:21:36.613199 27124 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486868287078-27124-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (351 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 22:21:36.614332 27124 mini_oidc.cc:152] Starting JWKS server
I20260811 22:21:36.615263 27124 webserver.cc:460] Webserver started at https://127.0.0.1:34947/ using document root <none> and password file <none>
I20260811 22:21:36.615381 27124 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:21:36.615629 27124 webserver.cc:460] Webserver started at http://127.0.0.1:36801/ using document root <none> and password file <none>
2026-08-11T22:21:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:36Z Disabled control of system clock
I20260811 22:21:36.626166 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:34909
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:34371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:34909
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34947/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/testchainca.cert with env {}
W20260811 22:21:36.744081 3884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:36.744388 3884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:36.744431 3884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:36.748504 3884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:36.748579 3884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:36.748593 3884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:36.748608 3884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:36.748622 3884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:21:36.748637 3884 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:21:36.748649 3884 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34947/jwks.json
I20260811 22:21:36.754390 3884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:34371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:34909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:34909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34947/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:36.755553 3884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:36.756601 3884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:36.766004 3894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:36.766004 3891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:36.766073 3884 server_base.cc:1034] running on GCE node
W20260811 22:21:36.766004 3896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:36.767170 3884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:36.768438 3884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:36.769639 3884 hybrid_clock.cc:648] HybridClock initialized: now 1786486896769612 us; error 40 us; skew 500 ppm
I20260811 22:21:36.771893 3884 webserver.cc:460] Webserver started at http://127.26.125.62:46603/ using document root <none> and password file <none>
I20260811 22:21:36.772547 3884 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:36.772620 3884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:36.772825 3884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:36.775203 3884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "124ea015e97f409fb9aa785acca1ce63"
format_stamp: "Formatted at 2026-08-11 22:21:36 on dist-test-slave-5bn2"
server_key: "00e52de7b51bc567f4782f9554ce21bf"
server_key_iv: "838c42ffd604e2209e6a67b736b74ccc"
server_key_version: "encryptionkey@0"
I20260811 22:21:36.775867 3884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "124ea015e97f409fb9aa785acca1ce63"
format_stamp: "Formatted at 2026-08-11 22:21:36 on dist-test-slave-5bn2"
server_key: "00e52de7b51bc567f4782f9554ce21bf"
server_key_iv: "838c42ffd604e2209e6a67b736b74ccc"
server_key_version: "encryptionkey@0"
I20260811 22:21:36.780086 3884 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:21:36.783083 3912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:36.784699 3884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:36.784853 3884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "124ea015e97f409fb9aa785acca1ce63"
format_stamp: "Formatted at 2026-08-11 22:21:36 on dist-test-slave-5bn2"
server_key: "00e52de7b51bc567f4782f9554ce21bf"
server_key_iv: "838c42ffd604e2209e6a67b736b74ccc"
server_key_version: "encryptionkey@0"
I20260811 22:21:36.785003 3884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:36.802834 3884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:36.803676 3884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:36.803877 3884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:36.810724 3884 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:34909
I20260811 22:21:36.810735 3990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:34909 every 8 connection(s)
I20260811 22:21:36.811787 3884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:36.812997 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 3884
I20260811 22:21:36.813140 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:36.813455 27124 external_mini_cluster.cc:1428] Setting key 2acf07cd9f31ef4dde5205bf7ee40b95
I20260811 22:21:36.815852 3992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:36.821519 3992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63: Bootstrap starting.
I20260811 22:21:36.823019 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.814950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36474 (local address 127.26.125.62:34909)
0811 22:21:36.815775 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:36.815795 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:36.815835 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:21:36.816554 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:36.816575 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:36.816999 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:36.817330 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:36.817774 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.818751 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.819667 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.820137 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.820284 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:36.821140 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:36.821145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:36.821158 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:36.821557 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:36.821616 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:36.821637 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:36.821726 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:36.821919 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 22:21:36.822191 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":133,"threads_started":1}
I20260811 22:21:36.823874 3992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:36.824760 3992 log.cc:824] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:36.825925 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:21:36.826778 3992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63: No bootstrap required, opened a new log
I20260811 22:21:36.830636 3992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124ea015e97f409fb9aa785acca1ce63" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34909 } }
I20260811 22:21:36.830927 3992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:36.830981 3992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 124ea015e97f409fb9aa785acca1ce63, State: Initialized, Role: FOLLOWER
I20260811 22:21:36.831384 3992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124ea015e97f409fb9aa785acca1ce63" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34909 } }
I20260811 22:21:36.831502 3992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:36.831555 3992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:36.831631 3992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:36.832572 3992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124ea015e97f409fb9aa785acca1ce63" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34909 } }
I20260811 22:21:36.832891 3992 leader_election.cc:302] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 124ea015e97f409fb9aa785acca1ce63; no voters:
I20260811 22:21:36.833174 3992 leader_election.cc:288] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:36.833290 3997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:36.833530 3997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [term 1 LEADER]: Becoming Leader. State: Replica: 124ea015e97f409fb9aa785acca1ce63, State: Running, Role: LEADER
I20260811 22:21:36.833879 3997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124ea015e97f409fb9aa785acca1ce63" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34909 } }
I20260811 22:21:36.834535 3992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:36.835810 3998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "124ea015e97f409fb9aa785acca1ce63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124ea015e97f409fb9aa785acca1ce63" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34909 } } }
I20260811 22:21:36.835817 4000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 124ea015e97f409fb9aa785acca1ce63. Latest consensus state: current_term: 1 leader_uuid: "124ea015e97f409fb9aa785acca1ce63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124ea015e97f409fb9aa785acca1ce63" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 34909 } } }
I20260811 22:21:36.835968 4000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:36.835968 3998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:36.836411 4009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:36.839696 4009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:36.845180 4009 catalog_manager.cc:1347] Generated new cluster ID: 919ca92dc39342bcad76f65e12bf1454
I20260811 22:21:36.845249 4009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:36.875360 4009 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:36.876396 4009 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:36.891963 4009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 124ea015e97f409fb9aa785acca1ce63: Generated new TSK 0
I20260811 22:21:36.892835 4009 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:21:36.915268 4039 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 22:21:36.915382 4039 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:21:36.916205 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:36.900397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36478 (local address 127.26.125.62:34909)
0811 22:21:36.900516 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:36.900520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:36.900771 (+ 251us) server_negotiation.cc:407] Connection header received
0811 22:21:36.900964 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:36.900966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:36.901019 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:36.901124 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:21:36.901386 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.901895 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.902793 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.903090 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.903185 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:36.903519 (+ 334us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:21:36.915707 (+ 12188us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:21:36.916041 (+ 334us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 22:21:36.916286 3995 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 22:21:36.916342 4036 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:36.900324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:34909 (local address 127.0.0.1:36478)
0811 22:21:36.900675 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:36.900689 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:36.900811 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:36.901135 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:36.901139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:36.901154 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:21:36.901281 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:36.901287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.902025 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.902029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:36.902675 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:36.902684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:36.903217 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:36.903221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:36.903311 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:36.903337 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:21:36.915975 (+ 12638us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:21:36.916171 (+ 196us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.125.62:34909: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":125,"threads_started":1}
I20260811 22:21:36.916788 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 3884
2026-08-11T22:21:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (373 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 22:21:36.988029 27124 mini_oidc.cc:152] Starting JWKS server
I20260811 22:21:36.988900 27124 webserver.cc:460] Webserver started at https://127.0.0.1:35625/ using document root <none> and password file <none>
I20260811 22:21:36.988998 27124 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:21:36.989219 27124 webserver.cc:460] Webserver started at http://127.0.0.1:34539/ using document root <none> and password file <none>
2026-08-11T22:21:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:36Z Disabled control of system clock
I20260811 22:21:37.000021 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36555
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36555
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35625/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:21:37.119968 4047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:37.120262 4047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:37.120302 4047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:37.124357 4047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:37.124424 4047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:37.124439 4047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:37.124454 4047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:37.124466 4047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:21:37.124480 4047 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:21:37.124500 4047 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35625/jwks.json
I20260811 22:21:37.129369 4047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35625/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:37.130477 4047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:37.131908 4047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:37.140661 4057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:37.140663 4060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:37.140663 4056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:37.140930 4047 server_base.cc:1034] running on GCE node
I20260811 22:21:37.141678 4047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:37.142967 4047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:37.144140 4047 hybrid_clock.cc:648] HybridClock initialized: now 1786486897144108 us; error 46 us; skew 500 ppm
I20260811 22:21:37.146288 4047 webserver.cc:460] Webserver started at http://127.26.125.62:34957/ using document root <none> and password file <none>
I20260811 22:21:37.146942 4047 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:37.147018 4047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:37.147254 4047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:37.149451 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "6d936346b2244e2383b3ba983e456983"
format_stamp: "Formatted at 2026-08-11 22:21:37 on dist-test-slave-5bn2"
server_key: "acf5f45f99f9926079baf3b1afcbd59d"
server_key_iv: "b0aa004a789f5066dd4a7eec4eb1864f"
server_key_version: "encryptionkey@0"
I20260811 22:21:37.150043 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "6d936346b2244e2383b3ba983e456983"
format_stamp: "Formatted at 2026-08-11 22:21:37 on dist-test-slave-5bn2"
server_key: "acf5f45f99f9926079baf3b1afcbd59d"
server_key_iv: "b0aa004a789f5066dd4a7eec4eb1864f"
server_key_version: "encryptionkey@0"
I20260811 22:21:37.154213 4047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 22:21:37.156976 4071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:37.158380 4047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:37.158525 4047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "6d936346b2244e2383b3ba983e456983"
format_stamp: "Formatted at 2026-08-11 22:21:37 on dist-test-slave-5bn2"
server_key: "acf5f45f99f9926079baf3b1afcbd59d"
server_key_iv: "b0aa004a789f5066dd4a7eec4eb1864f"
server_key_version: "encryptionkey@0"
I20260811 22:21:37.158658 4047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:37.180717 4047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:37.181466 4047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:37.181661 4047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:37.188728 4047 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:36555
I20260811 22:21:37.188740 4150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:36555 every 8 connection(s)
I20260811 22:21:37.189766 4047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:37.193035 4151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:37.197389 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 4047
I20260811 22:21:37.197541 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:37.197819 27124 external_mini_cluster.cc:1428] Setting key 86dfde75b3d3b84a5390d99b85e1ffb7
I20260811 22:21:37.199381 4151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983: Bootstrap starting.
I20260811 22:21:37.201946 4151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:37.202723 4151 log.cc:824] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:37.204767 4151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983: No bootstrap required, opened a new log
I20260811 22:21:37.207485 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:37.199383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.26.125.62:36555)
0811 22:21:37.200065 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:37.200082 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:37.200122 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:21:37.200800 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:37.200818 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:37.201253 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:37.201561 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:37.201951 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.203065 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.204013 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.204531 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.204688 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:37.205563 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:37.205568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:37.205584 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:37.206020 (+ 436us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:37.206078 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:37.206100 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:37.206189 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:37.206329 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 22:21:37.206619 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":127,"threads_started":1}
I20260811 22:21:37.208786 4151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d936346b2244e2383b3ba983e456983" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36555 } }
I20260811 22:21:37.209033 4151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:37.209101 4151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d936346b2244e2383b3ba983e456983, State: Initialized, Role: FOLLOWER
I20260811 22:21:37.209556 4151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d936346b2244e2383b3ba983e456983" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36555 } }
I20260811 22:21:37.209722 4151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:37.209789 4151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:37.209889 4151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:37.210686 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:21:37.210942 4151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d936346b2244e2383b3ba983e456983" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36555 } }
I20260811 22:21:37.211334 4151 leader_election.cc:302] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d936346b2244e2383b3ba983e456983; no voters:
I20260811 22:21:37.211704 4151 leader_election.cc:288] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:37.211808 4156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:37.212148 4156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [term 1 LEADER]: Becoming Leader. State: Replica: 6d936346b2244e2383b3ba983e456983, State: Running, Role: LEADER
I20260811 22:21:37.212460 4156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d936346b2244e2383b3ba983e456983" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36555 } }
I20260811 22:21:37.213286 4151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:37.213701 4160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d936346b2244e2383b3ba983e456983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d936346b2244e2383b3ba983e456983" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36555 } } }
I20260811 22:21:37.213860 4160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:37.214156 4159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d936346b2244e2383b3ba983e456983. Latest consensus state: current_term: 1 leader_uuid: "6d936346b2244e2383b3ba983e456983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d936346b2244e2383b3ba983e456983" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36555 } } }
I20260811 22:21:37.214255 4159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:37.214527 4163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:37.217868 4163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:37.223950 4163 catalog_manager.cc:1347] Generated new cluster ID: 8d57a2d62a354c2fac8447cfc47f2013
I20260811 22:21:37.224035 4163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:37.232717 4163 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:37.233722 4163 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:37.244251 4163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d936346b2244e2383b3ba983e456983: Generated new TSK 0
I20260811 22:21:37.245298 4163 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:21:37.266528 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:37.261104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.26.125.62:36555)
0811 22:21:37.261253 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:37.261258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:37.261443 (+ 185us) server_negotiation.cc:407] Connection header received
0811 22:21:37.261635 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:37.261638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:37.261691 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:37.261787 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:21:37.262065 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.262674 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.263965 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.264206 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.264289 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:37.264651 (+ 362us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:21:37.266111 (+ 1460us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:21:37.266363 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33150: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:21:37.266610 4154 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33150: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 22:21:37.268393 4199 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:37.261004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36555 (local address 127.0.0.1:33150)
0811 22:21:37.261348 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:37.261364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:37.261497 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:37.261808 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:37.261811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:37.261820 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:21:37.261953 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:37.261960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.262784 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.262788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:37.263836 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:37.263849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.264320 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.264324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:37.264420 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:37.264455 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:21:37.266286 (+ 1831us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:21:37.268198 (+ 1912us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.125.62:36555: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":1663,"thread_start_us":103,"threads_started":1}
I20260811 22:21:37.269070 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 4047
E20260811 22:21:37.285552 4201 webserver.cc:533] Webserver: sslize failed:
E20260811 22:21:37.285650 4201 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T22:21:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (329 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T22:21:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:37Z Disabled control of system clock
I20260811 22:21:37.327316 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:32907
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:40579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:32907
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/testchainca.cert with env {}
W20260811 22:21:37.443871 4207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:37.444157 4207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:37.444197 4207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:37.448382 4207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:37.448454 4207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:37.448468 4207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:37.448483 4207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:37.448498 4207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:21:37.448523 4207 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:21:37.448549 4207 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 22:21:37.454079 4207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:32907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:37.455302 4207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:37.456379 4207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:37.465260 4215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:37.465260 4212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:37.465283 4213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:37.465705 4207 server_base.cc:1034] running on GCE node
I20260811 22:21:37.466324 4207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:37.467607 4207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:37.468811 4207 hybrid_clock.cc:648] HybridClock initialized: now 1786486897468783 us; error 49 us; skew 500 ppm
I20260811 22:21:37.470911 4207 webserver.cc:460] Webserver started at http://127.26.125.62:43049/ using document root <none> and password file <none>
I20260811 22:21:37.471511 4207 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:37.471585 4207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:37.471807 4207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:37.473948 4207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "9f1d98d15ff249668b42663fc1a1d9a9"
format_stamp: "Formatted at 2026-08-11 22:21:37 on dist-test-slave-5bn2"
server_key: "9cd0edfba78d6813567c6bf0c8d0db0c"
server_key_iv: "b70949b569aa81cc85861e392e649dfd"
server_key_version: "encryptionkey@0"
I20260811 22:21:37.474519 4207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "9f1d98d15ff249668b42663fc1a1d9a9"
format_stamp: "Formatted at 2026-08-11 22:21:37 on dist-test-slave-5bn2"
server_key: "9cd0edfba78d6813567c6bf0c8d0db0c"
server_key_iv: "b70949b569aa81cc85861e392e649dfd"
server_key_version: "encryptionkey@0"
I20260811 22:21:37.478328 4207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:37.480988 4234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:37.482339 4207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:21:37.482455 4207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "9f1d98d15ff249668b42663fc1a1d9a9"
format_stamp: "Formatted at 2026-08-11 22:21:37 on dist-test-slave-5bn2"
server_key: "9cd0edfba78d6813567c6bf0c8d0db0c"
server_key_iv: "b70949b569aa81cc85861e392e649dfd"
server_key_version: "encryptionkey@0"
I20260811 22:21:37.482580 4207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:37.497195 4207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:37.497982 4207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:37.498173 4207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:37.505283 4207 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:32907
I20260811 22:21:37.505292 4315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:32907 every 8 connection(s)
I20260811 22:21:37.506353 4207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:37.509548 4316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:37.514394 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 4207
I20260811 22:21:37.514566 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:37.514894 27124 external_mini_cluster.cc:1428] Setting key b6fac7d18da742397c5641dae2faf126
I20260811 22:21:37.516433 4316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9: Bootstrap starting.
I20260811 22:21:37.519148 4316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:37.519907 4316 log.cc:824] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:37.522097 4316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9: No bootstrap required, opened a new log
I20260811 22:21:37.524935 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:37.516425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42158 (local address 127.26.125.62:32907)
0811 22:21:37.517172 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:37.517191 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:37.517231 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:21:37.517931 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:37.517951 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:37.518386 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:37.518685 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:37.519112 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.520137 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.521234 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.521720 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.521839 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:37.522742 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:37.522747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:37.522762 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:21:37.523234 (+ 472us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:37.523292 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:21:37.523312 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:37.523388 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:37.523604 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 22:21:37.523888 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":151,"threads_started":1}
I20260811 22:21:37.525446 4316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1d98d15ff249668b42663fc1a1d9a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 32907 } }
I20260811 22:21:37.525678 4316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:37.525717 4316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1d98d15ff249668b42663fc1a1d9a9, State: Initialized, Role: FOLLOWER
I20260811 22:21:37.526245 4316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1d98d15ff249668b42663fc1a1d9a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 32907 } }
I20260811 22:21:37.526413 4316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:37.526475 4316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:37.526573 4316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:37.528048 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:21:37.529600 4316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1d98d15ff249668b42663fc1a1d9a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 32907 } }
I20260811 22:21:37.530049 4316 leader_election.cc:302] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f1d98d15ff249668b42663fc1a1d9a9; no voters:
I20260811 22:21:37.530435 4316 leader_election.cc:288] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:37.530560 4325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:37.530805 4325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1d98d15ff249668b42663fc1a1d9a9, State: Running, Role: LEADER
I20260811 22:21:37.531194 4325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1d98d15ff249668b42663fc1a1d9a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 32907 } }
I20260811 22:21:37.531831 4316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:37.532938 4329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f1d98d15ff249668b42663fc1a1d9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1d98d15ff249668b42663fc1a1d9a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 32907 } } }
I20260811 22:21:37.533083 4329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:37.533324 4329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f1d98d15ff249668b42663fc1a1d9a9. Latest consensus state: current_term: 1 leader_uuid: "9f1d98d15ff249668b42663fc1a1d9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1d98d15ff249668b42663fc1a1d9a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 32907 } } }
I20260811 22:21:37.533416 4329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:37.533919 4345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:37.537744 4345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:37.543807 4345 catalog_manager.cc:1347] Generated new cluster ID: 58114147d38b45b6b4e88f4bb1f1b9d4
I20260811 22:21:37.543905 4345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:37.553409 4345 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:37.554401 4345 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:37.560303 4345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f1d98d15ff249668b42663fc1a1d9a9: Generated new TSK 0
I20260811 22:21:37.561055 4345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:21:38.000869 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:37.572543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.26.125.62:32907)
0811 22:21:37.572677 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:37.572682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:37.572805 (+ 123us) server_negotiation.cc:407] Connection header received
0811 22:21:37.572966 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:37.572970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:37.573033 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:37.573164 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:21:37.573354 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.574191 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.575073 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.575427 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.575536 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:37.575827 (+ 291us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:21:38.000336 (+424509us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:21:38.000710 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42164: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 22:21:38.000974 4321 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42164: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 22:21:38.001032 4367 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:37.572447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:32907 (local address 127.0.0.1:42164)
0811 22:21:37.572718 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:37.572730 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:37.572832 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:37.573149 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:37.573151 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:37.573157 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:21:37.573260 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:37.573265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.574293 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.574296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:37.574968 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:37.574977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:37.575564 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:37.575567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:37.575633 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:37.575657 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:21:38.000640 (+424983us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:21:38.000845 (+ 205us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.125.62:32907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":85,"threads_started":1}
I20260811 22:21:38.001725 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 4207
2026-08-11T22:21:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (717 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (133 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (130 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:38 dist-test-slave-5bn2 krb5kdc[4387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:38 dist-test-slave-5bn2 krb5kdc[4387](info): set up 2 sockets
Aug 11 22:21:38 dist-test-slave-5bn2 krb5kdc[4387](info): commencing operation
krb5kdc: starting...
W20260811 22:21:39.466292 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:39 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:39.605537 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:39911
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:39911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:39.723160 4405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:39.723459 4405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:39.723517 4405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:39.727624 4405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:39.727705 4405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:39.727731 4405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:39.727752 4405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:39.727780 4405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:39.733430 4405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:39911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:39.734607 4405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:39.735746 4405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:39.744428 4411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:39.744401 4410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:39.744609 4405 server_base.cc:1034] running on GCE node
W20260811 22:21:39.744403 4413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:39.745350 4405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:39.746658 4405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:39.747840 4405 hybrid_clock.cc:648] HybridClock initialized: now 1786486899747829 us; error 53 us; skew 500 ppm
Aug 11 22:21:39 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486899, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:39.750839 4405 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:39.751813 4405 webserver.cc:460] Webserver started at http://127.26.125.62:43829/ using document root <none> and password file <none>
I20260811 22:21:39.752370 4405 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:39.752446 4405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:39.752674 4405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:39.754813 4405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "911011ba3a604064ab07ff8653554104"
format_stamp: "Formatted at 2026-08-11 22:21:39 on dist-test-slave-5bn2"
server_key: "efb8d5246a60b82542e4ee1163324742"
server_key_iv: "f9df8a0e2768cee84f25974d33e91d1f"
server_key_version: "encryptionkey@0"
I20260811 22:21:39.755457 4405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "911011ba3a604064ab07ff8653554104"
format_stamp: "Formatted at 2026-08-11 22:21:39 on dist-test-slave-5bn2"
server_key: "efb8d5246a60b82542e4ee1163324742"
server_key_iv: "f9df8a0e2768cee84f25974d33e91d1f"
server_key_version: "encryptionkey@0"
I20260811 22:21:39.759713 4405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:21:39.762404 4428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:39.763895 4405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:21:39.764041 4405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "911011ba3a604064ab07ff8653554104"
format_stamp: "Formatted at 2026-08-11 22:21:39 on dist-test-slave-5bn2"
server_key: "efb8d5246a60b82542e4ee1163324742"
server_key_iv: "f9df8a0e2768cee84f25974d33e91d1f"
server_key_version: "encryptionkey@0"
I20260811 22:21:39.764163 4405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:39.790719 4405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:39.794231 4405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:39.794422 4405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:39.801682 4405 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:39911
I20260811 22:21:39.801752 4526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:39911 every 8 connection(s)
I20260811 22:21:39.802731 4405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:39.803577 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 4405
I20260811 22:21:39.803733 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:39.804090 27124 external_mini_cluster.cc:1428] Setting key c592ff0e404a920f68cec43b49186d68
I20260811 22:21:39.806608 4528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:39.813753 4528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104: Bootstrap starting.
Aug 11 22:21:39 dist-test-slave-5bn2 krb5kdc[4387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:39.816438 4528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:39.817346 4528 log.cc:824] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:39.819792 4528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104: No bootstrap required, opened a new log
I20260811 22:21:39.820116 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:39.805686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.26.125.62:39911)
0811 22:21:39.806174 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:39.806195 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:39.806238 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:21:39.807088 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:39.807110 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:39.807509 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:39.807823 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:39.808824 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:39.809953 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:39.811147 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:39.811722 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:39.811863 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:39.814971 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:39.814978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:39.814994 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:39.815037 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:39.816800 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:39.817337 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:39.817343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:39.817348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:39.817552 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:39.817868 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:39.817872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:39.817874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:39.818485 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:39.818638 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:39.818967 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 22:21:39.819276 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":157,"threads_started":1}
I20260811 22:21:39.823688 4528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911011ba3a604064ab07ff8653554104" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39911 } }
I20260811 22:21:39.823943 4528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:39.824016 4528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911011ba3a604064ab07ff8653554104, State: Initialized, Role: FOLLOWER
I20260811 22:21:39.824462 4528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911011ba3a604064ab07ff8653554104" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39911 } }
I20260811 22:21:39.824597 4528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:39.824662 4528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:39.824765 4528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:39.825737 4528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911011ba3a604064ab07ff8653554104" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39911 } }
I20260811 22:21:39.826092 4528 leader_election.cc:302] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 911011ba3a604064ab07ff8653554104; no voters:
I20260811 22:21:39.826424 4528 leader_election.cc:288] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:39.826692 4534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:39.826956 4534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [term 1 LEADER]: Becoming Leader. State: Replica: 911011ba3a604064ab07ff8653554104, State: Running, Role: LEADER
I20260811 22:21:39.827265 4534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911011ba3a604064ab07ff8653554104" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39911 } }
I20260811 22:21:39.827785 4528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:39.828974 4536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "911011ba3a604064ab07ff8653554104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911011ba3a604064ab07ff8653554104" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39911 } } }
I20260811 22:21:39.829111 4536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:39.829408 4538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 911011ba3a604064ab07ff8653554104. Latest consensus state: current_term: 1 leader_uuid: "911011ba3a604064ab07ff8653554104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911011ba3a604064ab07ff8653554104" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39911 } } }
I20260811 22:21:39.829504 4538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:39.829578 4549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:39.832921 4549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:39.839700 4549 catalog_manager.cc:1347] Generated new cluster ID: 6d158dd72fd1482ca84edfc220b0c049
I20260811 22:21:39.839792 4549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:39.850330 4549 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:39.851531 4549 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:39.862560 4549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 911011ba3a604064ab07ff8653554104: Generated new TSK 0
I20260811 22:21:39.863518 4549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:39.901504 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:39911
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:40.031291 4571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:40.031607 4571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:40.031654 4571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:40.035717 4571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:40.035794 4571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:40.035867 4571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:40.041342 4571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:39911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:40.042498 4571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:40.043639 4571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:40.054348 4582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:40.054348 4580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:40.054347 4579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:40.054442 4571 server_base.cc:1034] running on GCE node
I20260811 22:21:40.055209 4571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:40.055984 4571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:40.057176 4571 hybrid_clock.cc:648] HybridClock initialized: now 1786486900057155 us; error 37 us; skew 500 ppm
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.060220 4571 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:40.061206 4571 webserver.cc:460] Webserver started at http://127.26.125.1:39955/ using document root <none> and password file <none>
I20260811 22:21:40.061964 4571 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:40.062067 4571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:40.062292 4571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:40.064585 4571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "0c5a067647674ac4ab657e128f6e2d1f"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "4e47d753e06b1c5bdfe8bb6d70d17eaf"
server_key_iv: "a055673f69b26688c77ff196d7a013ae"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.065189 4571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "0c5a067647674ac4ab657e128f6e2d1f"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "4e47d753e06b1c5bdfe8bb6d70d17eaf"
server_key_iv: "a055673f69b26688c77ff196d7a013ae"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.069301 4571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:40.072093 4596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.073493 4571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:40.073627 4571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "0c5a067647674ac4ab657e128f6e2d1f"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "4e47d753e06b1c5bdfe8bb6d70d17eaf"
server_key_iv: "a055673f69b26688c77ff196d7a013ae"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.073737 4571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:40.090749 4571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:40.093935 4571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:40.094138 4571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:40.094785 4571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:40.095732 4571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:40.095783 4571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.095839 4571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:40.095867 4571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.104521 4571 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:45211
I20260811 22:21:40.104552 4806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:45211 every 8 connection(s)
I20260811 22:21:40.105485 4571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:40.110646 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 4571
I20260811 22:21:40.110772 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:40.111104 27124 external_mini_cluster.cc:1428] Setting key 646dfd79ca413671f5c291475afb5485
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:40.120416 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.107457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:47315 (local address 127.26.125.62:39911)
0811 22:21:40.107612 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.107617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.108573 (+ 956us) server_negotiation.cc:407] Connection header received
0811 22:21:40.109951 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.109954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.110001 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.110088 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.111482 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.112027 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.113335 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.113659 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.113763 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.116504 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:40.116509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:40.116512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:40.116542 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:40.118144 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.118706 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.118710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.118712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.118814 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.119179 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.119183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.119184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.119457 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:40.119565 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:40.120069 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 22:21:40.120223 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:21:40.121236 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.107932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:39911 (local address 127.26.125.1:47315)
0811 22:21:40.108409 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:40.108455 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:40.109746 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:40.110243 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:40.110250 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:40.110671 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:40.111280 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:40.111297 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.112156 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.112159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:40.113204 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:40.113215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.113871 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.113877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:40.114141 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.114790 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:40.114809 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:40.116308 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:40.118257 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:40.118262 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:40.118277 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:40.118597 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:40.118946 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:40.118948 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:40.118949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:40.119093 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:40.119554 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:40.119560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:40.119823 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 22:21:40.119991 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 22:21:40.120274 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:21:40.122728 4809 heartbeater.cc:347] Connected to a master server at 127.26.125.62:39911
I20260811 22:21:40.123041 4809 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:40.123607 4809 heartbeater.cc:511] Master 127.26.125.62:39911 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:21:40.125429 4452 ts_manager.cc:194] Registered new tserver with Master: 0c5a067647674ac4ab657e128f6e2d1f (127.26.125.1:45211)
I20260811 22:21:40.127060 4452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:47315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:40.142617 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:39911
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:40.259322 4817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:40.259629 4817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:40.259691 4817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:40.264194 4817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:40.264272 4817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:40.264365 4817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:40.270183 4817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:39911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:40.271485 4817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:40.272730 4817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:40.283784 4824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:40.283766 4825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:40.283773 4827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:40.284225 4817 server_base.cc:1034] running on GCE node
I20260811 22:21:40.284678 4817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:40.285352 4817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:40.286540 4817 hybrid_clock.cc:648] HybridClock initialized: now 1786486900286500 us; error 49 us; skew 500 ppm
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.292887 4817 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:40.293895 4817 webserver.cc:460] Webserver started at http://127.26.125.2:43793/ using document root <none> and password file <none>
I20260811 22:21:40.294487 4817 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:40.294557 4817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:40.294772 4817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:40.297086 4817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "c7a9f6b7c35546ddbe09a7315143b2ce"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "8986a21aa98a65aa0003214673a526c4"
server_key_iv: "ef681a3045506f9ba2c93bda77c6ecbe"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.297669 4817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "c7a9f6b7c35546ddbe09a7315143b2ce"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "8986a21aa98a65aa0003214673a526c4"
server_key_iv: "ef681a3045506f9ba2c93bda77c6ecbe"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.301757 4817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:40.304410 4841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.305882 4817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:40.306039 4817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "c7a9f6b7c35546ddbe09a7315143b2ce"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "8986a21aa98a65aa0003214673a526c4"
server_key_iv: "ef681a3045506f9ba2c93bda77c6ecbe"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.306154 4817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:40.336459 4817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:40.339627 4817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:40.339825 4817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:40.340518 4817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:40.341408 4817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:40.341461 4817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.341518 4817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:40.341543 4817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.350208 4817 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:43967
I20260811 22:21:40.350240 5029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:43967 every 8 connection(s)
I20260811 22:21:40.351269 4817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:40.360773 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 4817
I20260811 22:21:40.360934 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:40.361287 27124 external_mini_cluster.cc:1428] Setting key a3ac883083a04f802a290b6c598f0cee
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:40.366622 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.353313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:33927 (local address 127.26.125.62:39911)
0811 22:21:40.353498 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.353502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.354419 (+ 917us) server_negotiation.cc:407] Connection header received
0811 22:21:40.355894 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.355898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.355961 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.356036 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.357374 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.357937 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.359201 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.359438 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.359543 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.362454 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:40.362457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:40.362459 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:40.362486 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:40.363898 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.364558 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.364561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.364562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.364644 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.365040 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.365044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.365045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.365311 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:40.365497 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:40.366252 (+ 755us) server_negotiation.cc:299] Negotiation successful
0811 22:21:40.366401 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:21:40.367509 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.353651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:39911 (local address 127.26.125.2:33927)
0811 22:21:40.354185 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:40.354236 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:40.355654 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:40.356224 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:40.356232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:40.356670 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:40.357199 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:40.357215 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.358074 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.358077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:40.359082 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:40.359092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.359596 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.359601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:40.359784 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.360438 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:40.360467 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:40.362281 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:40.364050 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:40.364057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:40.364076 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:40.364450 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:40.364727 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:40.364729 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:40.364731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:40.364923 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:40.365486 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:40.365492 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:40.365959 (+ 467us) client_negotiation.cc:769] Sending connection context
0811 22:21:40.366149 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:21:40.366459 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":128,"threads_started":1}
I20260811 22:21:40.368736 5031 heartbeater.cc:347] Connected to a master server at 127.26.125.62:39911
I20260811 22:21:40.368994 5031 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:40.369508 5031 heartbeater.cc:511] Master 127.26.125.62:39911 requested a full tablet report, sending...
I20260811 22:21:40.370697 4452 ts_manager.cc:194] Registered new tserver with Master: c7a9f6b7c35546ddbe09a7315143b2ce (127.26.125.2:43967)
I20260811 22:21:40.371384 4452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:33927
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:40.392539 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:39911
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:40.512284 5036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:40.512609 5036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:40.512657 5036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:40.516878 5036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:40.516973 5036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:40.517037 5036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:40.522744 5036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:39911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:40.524055 5036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:40.525146 5036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:40.536105 5049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:40.536110 5051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:40.536110 5048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:40.536314 5036 server_base.cc:1034] running on GCE node
I20260811 22:21:40.536962 5036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:40.537712 5036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:40.538913 5036 hybrid_clock.cc:648] HybridClock initialized: now 1786486900538884 us; error 38 us; skew 500 ppm
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.541993 5036 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:40.543006 5036 webserver.cc:460] Webserver started at http://127.26.125.3:41397/ using document root <none> and password file <none>
I20260811 22:21:40.543598 5036 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:40.543663 5036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:40.543845 5036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:40.546017 5036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "3c29a82175fb41daa4405512ff28e90f"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "85fb6571c2feb07c1257c8f934bf2c88"
server_key_iv: "041807e09d4cc1907541142eee39750a"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.546571 5036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "3c29a82175fb41daa4405512ff28e90f"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "85fb6571c2feb07c1257c8f934bf2c88"
server_key_iv: "041807e09d4cc1907541142eee39750a"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.550647 5036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 22:21:40.553335 5060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.554766 5036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:40.554908 5036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "3c29a82175fb41daa4405512ff28e90f"
format_stamp: "Formatted at 2026-08-11 22:21:40 on dist-test-slave-5bn2"
server_key: "85fb6571c2feb07c1257c8f934bf2c88"
server_key_iv: "041807e09d4cc1907541142eee39750a"
server_key_version: "encryptionkey@0"
I20260811 22:21:40.555020 5036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:40.577224 5036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:40.580307 5036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:40.580503 5036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:40.581135 5036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:40.582003 5036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:40.582054 5036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.582095 5036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:40.582122 5036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:40.590703 5036 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:43517
I20260811 22:21:40.590754 5255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:43517 every 8 connection(s)
I20260811 22:21:40.591667 5036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:40.600800 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 5036
I20260811 22:21:40.600927 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:40.601230 27124 external_mini_cluster.cc:1428] Setting key afd14f5be8d49a56387de2d31e9506a2
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:40.606307 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.593540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:55193 (local address 127.26.125.62:39911)
0811 22:21:40.593681 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.593685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.594595 (+ 910us) server_negotiation.cc:407] Connection header received
0811 22:21:40.596077 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.596080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.596133 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.596211 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.597488 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.598040 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.599487 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.599718 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.599805 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.602688 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:40.602693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:40.602696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:40.602729 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:40.604288 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.604833 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.604835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.604837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.604912 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.605229 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.605231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.605232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.605412 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:40.605481 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:40.606074 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 22:21:40.606181 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:21:40.607348 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.593940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:39911 (local address 127.26.125.3:55193)
0811 22:21:40.594427 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:40.594475 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:40.595820 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:40.596383 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:40.596390 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:40.596780 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:40.597325 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:40.597340 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.598239 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.598242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:40.599353 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:40.599364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.599875 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.599879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:40.600039 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.600678 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:40.600698 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:40.602521 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:40.604418 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:40.604424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:40.604441 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:40.604747 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:40.604986 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:40.604988 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:40.604990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:40.605152 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:40.605537 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:40.605543 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:40.605850 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 22:21:40.606033 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 22:21:40.606347 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":115,"threads_started":1}
I20260811 22:21:40.608554 5256 heartbeater.cc:347] Connected to a master server at 127.26.125.62:39911
I20260811 22:21:40.608790 5256 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:40.609288 5256 heartbeater.cc:511] Master 127.26.125.62:39911 requested a full tablet report, sending...
I20260811 22:21:40.610394 4452 ts_manager.cc:194] Registered new tserver with Master: 3c29a82175fb41daa4405512ff28e90f (127.26.125.3:43517)
I20260811 22:21:40.611120 4452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:55193
I20260811 22:21:40.615859 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/test-admin.keytab.
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.633073 27124 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 22:21:40.636004 5264 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:40.634455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.26.125.1:45211)
0811 22:21:40.634755 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.634761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.634790 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:21:40.634917 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.634923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.635140 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.635268 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.635833 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33720: BlockingRecv error: recv got EOF from 127.0.0.1:33720 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":106,"threads_started":1}
W20260811 22:21:40.636087 27124 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.637091 27124 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.637913 27124 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:40.647322 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.638805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33732 (local address 127.26.125.1:45211)
0811 22:21:40.638948 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.638952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.638969 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:21:40.639126 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.639129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.639174 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.639239 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.639746 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.640485 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.641314 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.641712 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.641806 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.644094 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:40.644105 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:40.644125 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:40.644149 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:40.645554 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.645958 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.645965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.645967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.646050 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.646294 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.646297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.646299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.646594 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:40.646825 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:40.647020 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 22:21:40.647176 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 22:21:40.651691 5264 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:40.650607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.26.125.1:45211)
0811 22:21:40.650756 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.650760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.650810 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:21:40.651024 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.651028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.651091 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.651169 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.651543 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33734: BlockingRecv error: recv got EOF from 127.0.0.1:33734 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 22:21:40.651811 27124 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.652876 27124 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.653750 27124 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:40.662770 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.654693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.26.125.1:45211)
0811 22:21:40.654829 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.654833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.654926 (+ 93us) server_negotiation.cc:407] Connection header received
0811 22:21:40.655041 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.655043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.655082 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.655148 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.655604 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.656101 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.656964 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.657209 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.657288 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.659752 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:40.659755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:40.659757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:40.659782 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:40.661162 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.661559 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.661562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.661563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.661637 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.661957 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.661960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.661961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.662136 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:40.662258 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:40.662508 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:21:40.662607 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 22:21:40.665519 5264 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:40.664550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33752 (local address 127.26.125.1:45211)
0811 22:21:40.664657 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.664661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.664801 (+ 140us) server_negotiation.cc:407] Connection header received
0811 22:21:40.664905 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.664908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.664948 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.665009 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.665417 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33752: BlockingRecv error: recv got EOF from 127.0.0.1:33752 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 22:21:40.665741 27124 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486868287078-27124-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.666774 27124 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:40.667620 27124 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[4387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:21:40.675839 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:40.668391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.26.125.1:45211)
0811 22:21:40.668630 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:40.668633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:40.668645 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:21:40.668678 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:40.668680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:40.668712 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:40.668772 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:40.669204 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.669755 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.670645 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:40.670918 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:40.671007 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:40.673159 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:40.673162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:40.673165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:40.673190 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:40.674469 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.674810 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.674813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.674814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.674911 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:40.675147 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:40.675150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:40.675151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:40.675322 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:40.675437 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:40.675585 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 22:21:40.675677 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260811 22:21:40.676880 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 4571
I20260811 22:21:40.692544 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 4817
I20260811 22:21:40.706532 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 5036
I20260811 22:21:40.721495 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 4405
2026-08-11T22:21:40Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2457 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[5286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[5286](info): set up 2 sockets
Aug 11 22:21:40 dist-test-slave-5bn2 krb5kdc[5286](info): commencing operation
krb5kdc: starting...
W20260811 22:21:41.922119 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:42 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.62.keytab.
Entry for principal oryx/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.62.keytab.
I20260811 22:21:42.055686 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45967
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.62.keytab
--principal=oryx/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:45967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:42.172555 5309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:42.172876 5309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:42.172936 5309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:42.177079 5309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:42.177166 5309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:42.177193 5309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:42.177215 5309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:42.177235 5309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:42.182950 5309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:45967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.62.keytab
--principal=oryx/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:42.184114 5309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:42.185221 5309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:42.193917 5324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:42.193921 5318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:42.194115 5309 server_base.cc:1034] running on GCE node
W20260811 22:21:42.193913 5317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:42.194887 5309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:42.196099 5309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:42.197278 5309 hybrid_clock.cc:648] HybridClock initialized: now 1786486902197258 us; error 30 us; skew 500 ppm
Aug 11 22:21:42 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:42.200258 5309 init.cc:377] Logged in from keytab as oryx/127.26.125.62@KRBTEST.COM (short username oryx)
I20260811 22:21:42.201239 5309 webserver.cc:460] Webserver started at http://127.26.125.62:43611/ using document root <none> and password file <none>
I20260811 22:21:42.201819 5309 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:42.201891 5309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:42.202092 5309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:42.204275 5309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "bee3d139283d4c78966aa9c23c82c6ca"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "ee4ab79ee279895a9560b438794133e1"
server_key_iv: "1ef208bb0225c73765758daf81a86add"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.204856 5309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "bee3d139283d4c78966aa9c23c82c6ca"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "ee4ab79ee279895a9560b438794133e1"
server_key_iv: "1ef208bb0225c73765758daf81a86add"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.208773 5309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:42.211321 5339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.212617 5309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:21:42.212735 5309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "bee3d139283d4c78966aa9c23c82c6ca"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "ee4ab79ee279895a9560b438794133e1"
server_key_iv: "1ef208bb0225c73765758daf81a86add"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.212834 5309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:42.223434 5309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:42.226498 5309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:42.226681 5309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:42.233768 5309 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:45967
I20260811 22:21:42.234045 5433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:45967 every 8 connection(s)
I20260811 22:21:42.234899 5309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:42.237872 5434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:42.244031 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 5309
I20260811 22:21:42.244201 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:42.244457 5434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: Bootstrap starting.
I20260811 22:21:42.244585 27124 external_mini_cluster.cc:1428] Setting key c4609db4c853a370bf4a9e12536b19cb
I20260811 22:21:42.247395 5434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:42.248274 5434 log.cc:824] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:42.252585 5434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: No bootstrap required, opened a new log
Aug 11 22:21:42 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:42.257133 5434 raft_consensus.cc:348] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } }
I20260811 22:21:42.257434 5434 raft_consensus.cc:374] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:42.257475 5434 raft_consensus.cc:729] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee3d139283d4c78966aa9c23c82c6ca, State: Initialized, Role: FOLLOWER
I20260811 22:21:42.258010 5434 consensus_queue.cc:260] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } }
I20260811 22:21:42.258150 5434 raft_consensus.cc:388] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:42.258214 5434 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:42.258323 5434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:42.259429 5434 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } }
I20260811 22:21:42.259846 5434 leader_election.cc:302] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bee3d139283d4c78966aa9c23c82c6ca; no voters:
I20260811 22:21:42.260279 5434 leader_election.cc:288] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:42.260388 5442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:42.260569 5442 raft_consensus.cc:686] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 1 LEADER]: Becoming Leader. State: Replica: bee3d139283d4c78966aa9c23c82c6ca, State: Running, Role: LEADER
I20260811 22:21:42.260931 5442 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } }
I20260811 22:21:42.261216 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:42.246059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.26.125.62:45967)
0811 22:21:42.246557 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:42.246577 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:42.246619 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:21:42.247376 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:42.247402 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:42.247830 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:42.248176 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:42.249338 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.250494 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.251763 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.252321 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.252485 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:42.255564 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:42.255571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:42.255590 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:42.255628 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:42.257535 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:42.258074 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:42.258082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:42.258088 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:42.258273 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:42.258582 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:42.258586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:42.258588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:42.259158 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:42.259377 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:42.259696 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 22:21:42.260097 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":156,"threads_started":1}
I20260811 22:21:42.261799 5434 sys_catalog.cc:564] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:42.262339 5442 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee3d139283d4c78966aa9c23c82c6ca. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } }
I20260811 22:21:42.262472 5442 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:42.262778 5444 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } }
I20260811 22:21:42.262921 5444 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
W20260811 22:21:42.266992 5473 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:42.267079 5473 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:42.267139 5452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:42.267961 5452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:42.273903 5452 catalog_manager.cc:1347] Generated new cluster ID: 357565593f734e1ca5de1813dad5b7a2
I20260811 22:21:42.273994 5452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:42.284587 5452 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:42.286019 5452 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:42.304123 5452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: Generated new TSK 0
I20260811 22:21:42.305105 5452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab.
Entry for principal oryx/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab.
I20260811 22:21:42.345808 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45967
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab
--principal=oryx/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:42.464357 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:42.464634 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:42.464675 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:42.468786 5485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:42.468861 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:42.468920 5485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:42.474299 5485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab
--principal=oryx/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:42.475412 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:42.476480 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:42.486521 5498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:42.486639 5485 server_base.cc:1034] running on GCE node
W20260811 22:21:42.486518 5493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:42.486518 5494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:42.487344 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:42.488083 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:42.489250 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786486902489220 us; error 41 us; skew 500 ppm
Aug 11 22:21:42 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:42.492312 5485 init.cc:377] Logged in from keytab as oryx/127.26.125.1@KRBTEST.COM (short username oryx)
I20260811 22:21:42.493316 5485 webserver.cc:460] Webserver started at http://127.26.125.1:41191/ using document root <none> and password file <none>
I20260811 22:21:42.493916 5485 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:42.493989 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:42.494220 5485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:42.496443 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "bde0b8d5e9a74062ba71967366d77a28"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "ab68ad5c9172dc3fd2541aeb809ebebc"
server_key_iv: "3bbbae982c93b51846ea4318817ae2d7"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.497036 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "bde0b8d5e9a74062ba71967366d77a28"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "ab68ad5c9172dc3fd2541aeb809ebebc"
server_key_iv: "3bbbae982c93b51846ea4318817ae2d7"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.501252 5485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:42.504128 5511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.505826 5485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:42.505986 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "bde0b8d5e9a74062ba71967366d77a28"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "ab68ad5c9172dc3fd2541aeb809ebebc"
server_key_iv: "3bbbae982c93b51846ea4318817ae2d7"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.506104 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:42.527160 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:42.530930 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:42.531124 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:42.531744 5485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:42.532692 5485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:42.532747 5485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.532793 5485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:42.532807 5485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.541931 5485 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:34713
I20260811 22:21:42.541961 5698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:34713 every 8 connection(s)
I20260811 22:21:42.543025 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:42.544137 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 5485
I20260811 22:21:42.544283 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:42.544699 27124 external_mini_cluster.cc:1428] Setting key 81428776bb58f615f87e30c1aab49496
WARNING: no policy specified for oryx/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.2@KRBTEST.COM" created.
Aug 11 22:21:42 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:42.564728 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:42.545242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:33917 (local address 127.26.125.62:45967)
0811 22:21:42.545371 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:42.545375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:42.546421 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 22:21:42.547887 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:42.547890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:42.547948 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:42.548025 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:42.549299 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.549828 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.551130 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.551503 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.551639 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:42.559634 (+ 7995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:42.559640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:42.559643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:42.559685 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:42.561538 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:42.562533 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:42.562538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:42.562539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:42.562651 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:42.563171 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:42.563175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:42.563177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:42.563501 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:42.563609 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:42.564343 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 22:21:42.564525 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:21:42.565546 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:42.545697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.1:33917)
0811 22:21:42.546262 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:42.546309 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:42.547651 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:42.548233 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:42.548240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:42.548636 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:42.549151 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:42.549165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.549956 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.549960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:42.550981 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:42.550994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.551693 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.551696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:42.551863 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:42.556153 (+ 4290us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:42.556178 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:42.559331 (+ 3153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:42.561771 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:42.561781 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:42.561806 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:42.562315 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:42.562781 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:42.562784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:42.562786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:42.563048 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:42.563630 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:42.563639 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:42.563992 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 22:21:42.564206 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 22:21:42.564578 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":159,"threads_started":1}
I20260811 22:21:42.567165 5699 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45967
I20260811 22:21:42.567449 5699 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:42.568022 5699 heartbeater.cc:511] Master 127.26.125.62:45967 requested a full tablet report, sending...
I20260811 22:21:42.569716 5374 ts_manager.cc:194] Registered new tserver with Master: bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713)
I20260811 22:21:42.570988 5374 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:33917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab.
Entry for principal oryx/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab.
I20260811 22:21:42.578190 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45967
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab
--principal=oryx/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:42.696748 5707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:42.697041 5707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:42.697085 5707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:42.701195 5707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:42.701268 5707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:42.701331 5707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:42.706910 5707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab
--principal=oryx/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:42.708078 5707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:42.709148 5707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:42.719302 5707 server_base.cc:1034] running on GCE node
W20260811 22:21:42.719269 5719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:42.719326 5718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:42.719326 5722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:42.719966 5707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:42.720597 5707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:42.721758 5707 hybrid_clock.cc:648] HybridClock initialized: now 1786486902721724 us; error 46 us; skew 500 ppm
Aug 11 22:21:42 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:42.725566 5707 init.cc:377] Logged in from keytab as oryx/127.26.125.2@KRBTEST.COM (short username oryx)
I20260811 22:21:42.726526 5707 webserver.cc:460] Webserver started at http://127.26.125.2:37373/ using document root <none> and password file <none>
I20260811 22:21:42.727150 5707 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:42.727223 5707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:42.727443 5707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:42.729728 5707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "13fbdc730e1446b8937ff734023d6305"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "8bdd5b863b234185359223cb02f402ec"
server_key_iv: "a700cb6d0e38d6eb8012753cc02c6048"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.730319 5707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "13fbdc730e1446b8937ff734023d6305"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "8bdd5b863b234185359223cb02f402ec"
server_key_iv: "a700cb6d0e38d6eb8012753cc02c6048"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.734117 5707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:42.736800 5732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.738175 5707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:42.738304 5707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "13fbdc730e1446b8937ff734023d6305"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "8bdd5b863b234185359223cb02f402ec"
server_key_iv: "a700cb6d0e38d6eb8012753cc02c6048"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.738415 5707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:42.752130 5707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:42.755591 5707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:42.755793 5707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:42.756424 5707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:42.757325 5707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:42.757375 5707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.757431 5707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:42.757457 5707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.766705 5707 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:34859
I20260811 22:21:42.766747 5933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:34859 every 8 connection(s)
I20260811 22:21:42.767735 5707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:42.776379 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 5707
I20260811 22:21:42.776503 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:42.776799 27124 external_mini_cluster.cc:1428] Setting key a1f771ac11096baf1fb809e128de28c6
Aug 11 22:21:42 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:42.782788 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:42.769737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:56917 (local address 127.26.125.62:45967)
0811 22:21:42.769887 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:42.769891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:42.770980 (+ 1089us) server_negotiation.cc:407] Connection header received
0811 22:21:42.772487 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:42.772491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:42.772556 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:42.772630 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:42.773987 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.774519 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.775735 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.775960 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.776050 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:42.778670 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:42.778674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:42.778677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:42.778708 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:42.780385 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:42.781077 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:42.781080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:42.781082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:42.781171 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:42.781556 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:42.781558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:42.781559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:42.781748 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:42.781824 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:42.782479 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 22:21:42.782617 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:42.783728 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:42.770223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.2:56917)
0811 22:21:42.770746 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:42.770794 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:42.772249 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:42.772839 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:42.772847 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:42.773258 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:42.773820 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:42.773838 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.774646 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.774649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:42.775608 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:42.775619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:42.776133 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:42.776139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:42.776295 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:42.776884 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:42.776908 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:42.778487 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:42.780539 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:42.780548 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:42.780570 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:42.780970 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:42.781255 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:42.781258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:42.781260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:42.781466 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:42.781843 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:42.781871 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:42.782219 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 22:21:42.782422 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:21:42.782791 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":134,"threads_started":1}
I20260811 22:21:42.785082 5936 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45967
I20260811 22:21:42.785405 5936 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:42.786108 5936 heartbeater.cc:511] Master 127.26.125.62:45967 requested a full tablet report, sending...
I20260811 22:21:42.787456 5374 ts_manager.cc:194] Registered new tserver with Master: 13fbdc730e1446b8937ff734023d6305 (127.26.125.2:34859)
WARNING: no policy specified for oryx/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:42.788275 5374 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:56917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab.
Entry for principal oryx/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab.
I20260811 22:21:42.809726 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45967
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab
--principal=oryx/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:42.928903 5942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:42.929216 5942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:42.929263 5942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:42.933393 5942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:42.933470 5942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:42.933531 5942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:42.939136 5942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab
--principal=oryx/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:42.940294 5942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:42.941339 5942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:42.951918 5952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:42.951915 5949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:42.951974 5942 server_base.cc:1034] running on GCE node
W20260811 22:21:42.951915 5948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:42.952863 5942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:42.953656 5942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:42.954843 5942 hybrid_clock.cc:648] HybridClock initialized: now 1786486902954819 us; error 39 us; skew 500 ppm
Aug 11 22:21:42 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:42.958146 5942 init.cc:377] Logged in from keytab as oryx/127.26.125.3@KRBTEST.COM (short username oryx)
I20260811 22:21:42.959338 5942 webserver.cc:460] Webserver started at http://127.26.125.3:45109/ using document root <none> and password file <none>
I20260811 22:21:42.960024 5942 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:42.960119 5942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:42.960377 5942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:42.962945 5942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "c84c5ba6d5184657bd4a9c17a5765a80"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "81448ec96f13417ca9346329c4f68595"
server_key_iv: "747718430291913484ddbea71f49498b"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.963627 5942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "c84c5ba6d5184657bd4a9c17a5765a80"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "81448ec96f13417ca9346329c4f68595"
server_key_iv: "747718430291913484ddbea71f49498b"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.968298 5942 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:21:42.971170 5964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.972643 5942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:42.972785 5942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "c84c5ba6d5184657bd4a9c17a5765a80"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "81448ec96f13417ca9346329c4f68595"
server_key_iv: "747718430291913484ddbea71f49498b"
server_key_version: "encryptionkey@0"
I20260811 22:21:42.972903 5942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:42.986793 5942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:42.991344 5942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:42.991577 5942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:42.992364 5942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:42.993489 5942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:42.993556 5942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:42.993625 5942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:42.993652 5942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.007678 5942 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:45661
I20260811 22:21:43.007722 6158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:45661 every 8 connection(s)
I20260811 22:21:43.008958 5942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:43.009519 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 5942
I20260811 22:21:43.009667 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:43.010011 27124 external_mini_cluster.cc:1428] Setting key ab6ea4e345396b56831e4903eedcafbf
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:43.028770 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.011551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:38573 (local address 127.26.125.62:45967)
0811 22:21:43.011687 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.011692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.012889 (+ 1197us) server_negotiation.cc:407] Connection header received
0811 22:21:43.014519 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.014522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.014575 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.014617 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.016341 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.017027 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.018842 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.019186 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.019298 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.022808 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.022813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.022816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.022883 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.024751 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.026981 (+ 2230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.026985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.026987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.027092 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.027554 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.027558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.027559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.027787 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.027852 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.028515 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.028639 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:21:43.030323 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.012148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.3:38573)
0811 22:21:43.012746 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.012795 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.014268 (+ 1473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.015028 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.015035 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.015454 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.016166 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.016189 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.017193 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.017197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.018681 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.018696 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.019368 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.019372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.019603 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.020324 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.020349 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.022538 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.024879 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.024887 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.024918 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.025344 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.027195 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.027198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.027199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.027431 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.027938 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.027945 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.028262 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.029045 (+ 783us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.029405 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":160,"threads_started":1}
I20260811 22:21:43.031728 6159 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45967
I20260811 22:21:43.032052 6159 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:43.032743 6159 heartbeater.cc:511] Master 127.26.125.62:45967 requested a full tablet report, sending...
I20260811 22:21:43.034072 5374 ts_manager.cc:194] Registered new tserver with Master: c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661)
I20260811 22:21:43.034905 5374 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:38573
I20260811 22:21:43.037184 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.125.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.61.keytab.
Entry for principal oryx/127.26.125.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.61.keytab.
I20260811 22:21:43.070420 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.61:36207
--webserver_interface=127.26.125.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.61.keytab
--principal=oryx/127.26.125.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:45967,127.26.125.61:36207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:43.192163 6176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:43.192467 6176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:43.192517 6176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:43.196599 6176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:43.196664 6176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:43.196683 6176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:43.196698 6176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:43.196712 6176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:43.202374 6176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:45967,127.26.125.61:36207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.61.keytab
--principal=oryx/127.26.125.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.61:36207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.125.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
W20260811 22:21:43.202639 6176 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.125.62:45967,127.26.125.61:36207'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 22:21:43.203641 6176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:43.204711 6176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:43.213959 6181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:43.213959 6183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:43.214003 6176 server_base.cc:1034] running on GCE node
W20260811 22:21:43.213972 6188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:43.214823 6176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:43.216118 6176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:43.217288 6176 hybrid_clock.cc:648] HybridClock initialized: now 1786486903217259 us; error 49 us; skew 500 ppm
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:43.220252 6176 init.cc:377] Logged in from keytab as oryx/127.26.125.61@KRBTEST.COM (short username oryx)
I20260811 22:21:43.221256 6176 webserver.cc:460] Webserver started at http://127.26.125.61:37483/ using document root <none> and password file <none>
I20260811 22:21:43.221825 6176 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:43.221887 6176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:43.222084 6176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:43.224392 6176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data/instance:
uuid: "2587ac2153bd4418bbb8b86a8184cde2"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "5bb83ec60b8723c9b71e56e18311f8dc"
server_key_iv: "676ca9b4adeab06a41ced3b520bd6f71"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.224978 6176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal/instance:
uuid: "2587ac2153bd4418bbb8b86a8184cde2"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "5bb83ec60b8723c9b71e56e18311f8dc"
server_key_iv: "676ca9b4adeab06a41ced3b520bd6f71"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.229112 6176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 22:21:43.232141 6204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.233609 6176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:21:43.233750 6176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
uuid: "2587ac2153bd4418bbb8b86a8184cde2"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "5bb83ec60b8723c9b71e56e18311f8dc"
server_key_iv: "676ca9b4adeab06a41ced3b520bd6f71"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.233875 6176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:43.248754 6176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:43.252264 6176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:43.252465 6176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:43.260008 6176 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.61:36207
I20260811 22:21:43.260304 6309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.61:36207 every 8 connection(s)
I20260811 22:21:43.261173 6176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
I20260811 22:21:43.264364 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:43.268455 6311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:43.268787 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 6176
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.61@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.125.61@KRBTEST.COM
I20260811 22:21:43.282187 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.270514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48238 (local address 127.26.125.62:45967)
0811 22:21:43.270656 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.270661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.271079 (+ 418us) server_negotiation.cc:407] Connection header received
0811 22:21:43.271912 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.271916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.271969 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.272072 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.272927 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.273451 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.274545 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.274761 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.274875 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.277969 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.277973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.277975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.278001 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.279841 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.280433 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.280437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.280439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.280538 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.280925 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.280928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.280930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.281193 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.281291 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.281884 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.282017 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:43.283190 6315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.270493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48238)
0811 22:21:43.270939 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.270982 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.271699 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.272128 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.272133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.272515 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.272781 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.272794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.273619 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.273622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.274418 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.274427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.274933 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.274937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.275151 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.276213 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.276235 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.277828 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.279972 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.279981 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.279992 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.280335 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.280628 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.280631 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.280632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.280827 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.281312 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.281319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.281634 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.281816 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.282193 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":64,"spinlock_wait_cycles":2816,"thread_start_us":135,"threads_started":1}
I20260811 22:21:43.283193 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.270379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54010 (local address 127.26.125.61:36207)
0811 22:21:43.270834 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.270880 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.270944 (+ 64us) server_negotiation.cc:407] Connection header received
0811 22:21:43.271682 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.271703 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.271915 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.272080 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.272916 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.273790 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.274637 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.275103 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.275196 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.278407 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.278415 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.278431 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.278454 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.280350 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.280857 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.280863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.280864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.280957 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.281257 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.281260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.281262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.281616 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.281782 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.282054 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.282249 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":307,"spinlock_wait_cycles":896,"thread_start_us":152,"threads_started":1}
I20260811 22:21:43.284672 6311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.61@KRBTEST.COM for oryx/127.26.125.61@KRBTEST.COM
I20260811 22:21:43.293265 6315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.285227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.0.0.1:54016)
0811 22:21:43.285335 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.285349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.285421 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.285649 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.285652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.285817 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.285909 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.285915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.286630 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.286633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.287194 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.287203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.288106 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.288109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.288229 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.288773 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.288785 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.290142 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.291815 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.291818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.291821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.292014 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.292256 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.292259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.292262 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.292336 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.292726 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.292729 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.292816 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.292899 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.293064 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":20}
I20260811 22:21:43.293265 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.285291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.26.125.61:36207)
0811 22:21:43.285394 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.285398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.285413 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:43.285513 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.285516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.285561 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.285636 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.286012 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.286530 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.287298 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.287992 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.288091 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.290270 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.290274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.290276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.290297 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.291712 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.292099 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.292103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.292105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.292179 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.292412 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.292415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.292417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.292640 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.292726 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.292953 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.293114 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":36}
I20260811 22:21:43.296803 6311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Bootstrap starting.
I20260811 22:21:43.299573 6311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:43.300376 6311 log.cc:824] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:43.302582 6311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: No bootstrap required, opened a new log
I20260811 22:21:43.306095 6311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } }
I20260811 22:21:43.306350 6311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:43.306386 6311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2587ac2153bd4418bbb8b86a8184cde2, State: Initialized, Role: FOLLOWER
I20260811 22:21:43.307046 6311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } }
I20260811 22:21:43.307737 6317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } } }
I20260811 22:21:43.307890 6317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:43.308565 6311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:21:43.313735 6337 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:43.313850 6337 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:43.316068 6176 master_runner.cc:428] Detected that this master 2587ac2153bd4418bbb8b86a8184cde2 is joining an existing cluster
I20260811 22:21:43.316141 6176 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.125.61:36207
I20260811 22:21:43.326988 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.318799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.0.0.1:54020)
0811 22:21:43.319196 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.319207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.319318 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.319520 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.319522 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.319754 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.319861 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.319867 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.320800 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.320804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.321386 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.321394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.322033 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.322036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.322229 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.322750 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.322765 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.323254 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.325165 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.325168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.325170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.325405 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.325751 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.325754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.325757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.325889 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.326354 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.326357 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.326475 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.326588 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.326743 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":67,"thread_start_us":137,"threads_started":1}
I20260811 22:21:43.327018 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.318602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.26.125.61:36207)
0811 22:21:43.318716 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.318721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.319277 (+ 556us) server_negotiation.cc:407] Connection header received
0811 22:21:43.319393 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.319396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.319439 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.319496 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.319976 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.320697 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.321541 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.321882 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.321990 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.323365 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.323368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.323371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.323395 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.325058 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.325510 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.325533 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.325535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.325637 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.325974 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.325978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.325979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.326237 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.326330 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.326646 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.326830 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":44}
I20260811 22:21:43.328641 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.318398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48246)
0811 22:21:43.319120 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.319137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.319260 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.320007 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.320009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.320228 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.320343 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.320350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.321479 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.321483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.322200 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.322211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.322767 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.322771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.322887 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.323643 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.323653 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.324067 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.327005 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.327008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.327009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.327272 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.327571 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.327574 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.327576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.327664 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.328119 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.328122 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.328219 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.328322 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.328472 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":34,"thread_start_us":135,"threads_started":1}
I20260811 22:21:43.328663 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.318466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.26.125.62:45967)
0811 22:21:43.318613 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.318618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.319227 (+ 609us) server_negotiation.cc:407] Connection header received
0811 22:21:43.319368 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.319371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.319426 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.319506 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.320464 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.321359 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.322316 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.322649 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.322747 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.324228 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.324232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.324235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.324264 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.326829 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.327357 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.327361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.327363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.327473 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.327768 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.327771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.327773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.328010 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.328118 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.328379 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.328506 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.62@KRBTEST.COM for oryx/127.26.125.61@KRBTEST.COM
I20260811 22:21:43.345566 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.333639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.26.125.61:36207)
0811 22:21:43.333767 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.333772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.334111 (+ 339us) server_negotiation.cc:407] Connection header received
0811 22:21:43.334341 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.334344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.334401 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.334472 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.335225 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.336032 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.337341 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.337659 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.337803 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.340630 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.340634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.340636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.340664 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.343431 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.344040 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.344044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.344046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.344153 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.344451 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.344455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.344456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.344718 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.344822 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.345237 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.345374 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:21:43.345570 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.333585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.0.0.1:54034)
0811 22:21:43.333980 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.334038 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.334181 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.334555 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.334561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.334993 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.335118 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.335125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.336177 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.336180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.337228 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.337240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.337776 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.337780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.337901 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.338525 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.338540 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.340461 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.343565 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.343577 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.343586 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.343929 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.344250 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.344253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.344255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.344359 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.344828 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.344836 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.344988 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.345157 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.345380 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for oryx/127.26.125.60@KRBTEST.COM; defaulting to no policy
I20260811 22:21:43.347640 5374 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.125.61:36207: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2"
member_type: NON_VOTER
last_known_addr {
host: "127.26.125.61"
port: 36207
}
attrs {
promote: true
}
}
dest_uuid: "bee3d139283d4c78966aa9c23c82c6ca"
cas_config_opid_index: -1
I20260811 22:21:43.348635 5374 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.60@KRBTEST.COM" created.
I20260811 22:21:43.351022 6356 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2587ac2153bd4418bbb8b86a8184cde2 (127.26.125.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } }
I20260811 22:21:43.352177 6259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:43.353220 6356 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.125.61:36207
I20260811 22:21:43.353549 6356 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } }
I20260811 22:21:43.353781 6356 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:43.353881 6176 master.cc:537] Master@127.26.125.61:36207 shutting down...
I20260811 22:21:43.354334 6259 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Refusing update from remote peer bee3d139283d4c78966aa9c23c82c6ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 22:21:43.355224 6357 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:21:43.357616 5346 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 22:21:43.358642 5346 consensus_peers.cc:487] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca -> Peer 2587ac2153bd4418bbb8b86a8184cde2 (127.26.125.61:36207): Couldn't send request to peer 2587ac2153bd4418bbb8b86a8184cde2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:21:43.361120 6176 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:43.361474 6176 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:43.361596 6176 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.60.keytab.
Entry for principal oryx/127.26.125.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.60.keytab.
I20260811 22:21:43.378183 6176 master.cc:552] Master@127.26.125.61:36207 shutdown complete.
I20260811 22:21:43.378247 6176 master_runner.cc:305] Clearing existing system tablet
I20260811 22:21:43.378527 6176 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:43.378599 6176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:43.379122 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.60:41171
--webserver_interface=127.26.125.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.60.keytab
--principal=oryx/127.26.125.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:45967,127.26.125.61:36207,127.26.125.60:41171
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:43.380061 6176 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:43.381085 6367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.381318 6176 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:43.381448 6176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:43.381488 6176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
uuid: "2587ac2153bd4418bbb8b86a8184cde2"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "5bb83ec60b8723c9b71e56e18311f8dc"
server_key_iv: "676ca9b4adeab06a41ced3b520bd6f71"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.383597 6176 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:43.386351 6176 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:21:43.386473 6176 log.cc:1196] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 22:21:43.386761 6176 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Deleting consensus metadata
I20260811 22:21:43.386984 6176 master_runner.cc:315] Copying system tablet from 127.26.125.62:45967
I20260811 22:21:43.388357 6176 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: tablet copy: Beginning tablet copy session from remote peer at address 127.26.125.62:45967
I20260811 22:21:43.398108 6385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.388993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48260)
0811 22:21:43.389349 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.389364 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.389494 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.389785 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.389788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.390026 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.390191 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.390198 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.391346 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.391349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.392304 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.392316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.393131 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.393135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.393235 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.393816 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.393831 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.394297 (+ 466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.396425 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.396429 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.396432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.396716 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.397000 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.397003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.397005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.397099 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.397507 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.397510 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.397629 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.397731 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.397908 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":103,"threads_started":1}
I20260811 22:21:43.398110 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.389123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.26.125.62:45967)
0811 22:21:43.389272 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.389277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.389459 (+ 182us) server_negotiation.cc:407] Connection header received
0811 22:21:43.389593 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.389597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.389656 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.389764 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.390321 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.391225 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.392477 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.392855 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.392933 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.394457 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.394461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.394464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.394495 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.396289 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.396813 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.396816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.396818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.396914 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.397180 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.397183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.397185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.397419 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.397517 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.397798 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.397937 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:21:43.399248 5405 tablet_copy_service.cc:140] P bee3d139283d4c78966aa9c23c82c6ca: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2587ac2153bd4418bbb8b86a8184cde2 ({username='oryx', principal='oryx/127.26.125.61@KRBTEST.COM'} at 127.0.0.1:48260)
I20260811 22:21:43.399454 5405 tablet_copy_service.cc:161] P bee3d139283d4c78966aa9c23c82c6ca: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2587ac2153bd4418bbb8b86a8184cde2 at {username='oryx', principal='oryx/127.26.125.61@KRBTEST.COM'} at 127.0.0.1:48260: session id = 2587ac2153bd4418bbb8b86a8184cde2-00000000000000000000000000000000
I20260811 22:21:43.401296 5405 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:21:43.403264 6176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:21:43.405748 6176 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: tablet copy: Starting download of 0 data blocks...
I20260811 22:21:43.406030 6176 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: tablet copy: Starting download of 1 WAL segments...
I20260811 22:21:43.407984 6176 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:21:43.410374 5405 tablet_copy_service.cc:342] P bee3d139283d4c78966aa9c23c82c6ca: Request end of tablet copy session 2587ac2153bd4418bbb8b86a8184cde2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.125.61@KRBTEST.COM'} at 127.0.0.1:48260
I20260811 22:21:43.410526 5405 tablet_copy_service.cc:433] P bee3d139283d4c78966aa9c23c82c6ca: ending tablet copy session 2587ac2153bd4418bbb8b86a8184cde2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2587ac2153bd4418bbb8b86a8184cde2
I20260811 22:21:43.419384 6176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:43.421257 6387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:43.421360 6394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:43.421528 6176 server_base.cc:1034] running on GCE node
W20260811 22:21:43.421604 6391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:43.421809 6176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:43.422140 6176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:43.423282 6176 hybrid_clock.cc:648] HybridClock initialized: now 1786486903423261 us; error 38 us; skew 500 ppm
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:43.424980 6176 init.cc:377] Logged in from keytab as oryx/127.26.125.61@KRBTEST.COM (short username oryx)
I20260811 22:21:43.425498 6176 webserver.cc:460] Webserver started at http://127.26.125.61:39661/ using document root <none> and password file <none>
I20260811 22:21:43.425688 6176 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:43.425729 6176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:43.427081 6176 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:43.428097 6404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.428375 6176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:43.428453 6176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
uuid: "2587ac2153bd4418bbb8b86a8184cde2"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "5bb83ec60b8723c9b71e56e18311f8dc"
server_key_iv: "676ca9b4adeab06a41ced3b520bd6f71"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.428534 6176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:43.442382 6176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:43.444901 6176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:43.450019 6176 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.61:36207
I20260811 22:21:43.450177 6509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.61:36207 every 8 connection(s)
I20260811 22:21:43.450644 6176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
I20260811 22:21:43.452745 6510 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:21:43.453938 6510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Bootstrap starting.
I20260811 22:21:43.465495 6510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:21:43.465884 6510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: Bootstrap complete.
I20260811 22:21:43.466621 6510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } }
I20260811 22:21:43.466806 6510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2587ac2153bd4418bbb8b86a8184cde2, State: Initialized, Role: LEARNER
I20260811 22:21:43.466972 6510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } }
I20260811 22:21:43.467370 6516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } }
I20260811 22:21:43.467517 6510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:43.467566 6516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:21:43.471474 6532 catalog_manager.cc:1259] Loaded cluster ID: 357565593f734e1ca5de1813dad5b7a2
I20260811 22:21:43.471550 6532 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: loading cluster ID for follower catalog manager: success
I20260811 22:21:43.473716 6532 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: acquiring CA information for follower catalog manager: success
I20260811 22:21:43.475247 6532 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:21:43.518666 6366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:43.519004 6366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:43.519055 6366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:43.523169 6366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:43.523240 6366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:43.523259 6366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:43.523275 6366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:43.523289 6366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:43.528970 6366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:45967,127.26.125.61:36207,127.26.125.60:41171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.60.keytab
--principal=oryx/127.26.125.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.60:41171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.125.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:43.530273 6366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:43.531512 6366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:43.540583 6542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:43.540594 6540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:43.540594 6546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:43.540831 6366 server_base.cc:1034] running on GCE node
I20260811 22:21:43.541580 6366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:43.543164 6366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:43.544370 6366 hybrid_clock.cc:648] HybridClock initialized: now 1786486903544351 us; error 40 us; skew 500 ppm
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:43.547765 6366 init.cc:377] Logged in from keytab as oryx/127.26.125.60@KRBTEST.COM (short username oryx)
I20260811 22:21:43.548785 6366 webserver.cc:460] Webserver started at http://127.26.125.60:38173/ using document root <none> and password file <none>
I20260811 22:21:43.549333 6366 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:43.549394 6366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:43.549577 6366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:43.551776 6366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data/instance:
uuid: "e8150f1dce374c6c979f79e3220374ff"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "7d4a62370dd50d5ec8d7d5e12bef8d45"
server_key_iv: "58d260c4fc29b5cd58dc7ff7ed02acbb"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.552320 6366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal/instance:
uuid: "e8150f1dce374c6c979f79e3220374ff"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "7d4a62370dd50d5ec8d7d5e12bef8d45"
server_key_iv: "58d260c4fc29b5cd58dc7ff7ed02acbb"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.556346 6366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:43.559260 6557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.560913 6366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:21:43.561087 6366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
uuid: "e8150f1dce374c6c979f79e3220374ff"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "7d4a62370dd50d5ec8d7d5e12bef8d45"
server_key_iv: "58d260c4fc29b5cd58dc7ff7ed02acbb"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.561228 6366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:43.573542 5699 heartbeater.cc:503] Master 127.26.125.62:45967 was elected leader, sending a full tablet report...
I20260811 22:21:43.577064 6366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:43.580737 6366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:43.580945 6366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:43.588738 6366 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.60:41171
I20260811 22:21:43.589073 6659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.60:41171 every 8 connection(s)
I20260811 22:21:43.589849 6366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
I20260811 22:21:43.593027 6661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:43.597229 6661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:43.598573 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 6366
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.60@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.125.60@KRBTEST.COM
I20260811 22:21:43.611722 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.599415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48262 (local address 127.26.125.62:45967)
0811 22:21:43.599566 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.599571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.600483 (+ 912us) server_negotiation.cc:407] Connection header received
0811 22:21:43.601579 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.601582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.601634 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.601706 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.602715 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.603294 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.604433 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.604712 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.604791 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.607742 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.607746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.607749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.607783 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.609348 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.609984 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.609988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.609990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.610102 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.610534 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.610538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.610540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.610739 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.610893 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.611417 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.611547 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:21:43.612694 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.599801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48262)
0811 22:21:43.600325 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.600374 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.601343 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.601783 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.601789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.602290 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.602559 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.602578 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.603422 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.603426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.604310 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.604320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.604851 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.604855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.605103 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.605826 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.605843 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.607545 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.609463 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.609471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.609484 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.609842 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.610228 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.610231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.610233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.610382 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.610868 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.610876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.611178 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.611355 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.611714 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":25,"thread_start_us":135,"threads_started":1}
I20260811 22:21:43.612699 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.600663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.26.125.60:41171)
0811 22:21:43.600963 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.600974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.601022 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:21:43.601347 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.601372 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.601640 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.601822 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.602599 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.603667 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.604529 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.605034 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.605130 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.608135 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.608143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.608158 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.608183 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.610103 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.610560 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.610567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.610569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.610650 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.610969 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.610973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.610974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.611244 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.611451 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.611667 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.611948 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260811 22:21:43.614569 6661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.60@KRBTEST.COM for oryx/127.26.125.61@KRBTEST.COM
I20260811 22:21:43.623085 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.615089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.0.0.1:54042)
0811 22:21:43.615219 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.615233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.615310 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.615726 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.615728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.615876 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.615988 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.615993 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.617029 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.617032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.617672 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.617683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.618131 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.618134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.618221 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.618703 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.618715 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.619994 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.621493 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.621496 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.621497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.621728 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.622026 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.622028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.622030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.622128 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.622572 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.622575 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.622683 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.622772 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.622936 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:21:43.623231 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.615215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.26.125.61:36207)
0811 22:21:43.615534 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.615538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.615552 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:43.615589 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.615591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.615632 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.615705 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.616073 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.616898 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.617789 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.618038 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.618120 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.620136 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.620139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.620141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.620169 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.621380 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.621802 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.621804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.621805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.621886 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.622215 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.622217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.622219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.622414 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.622526 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.622946 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.623085 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
I20260811 22:21:43.624146 6661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.60@KRBTEST.COM for oryx/127.26.125.60@KRBTEST.COM
I20260811 22:21:43.631811 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.624641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.0.0.1:42184)
0811 22:21:43.624759 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.624773 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.624835 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.625031 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.625033 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.625169 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.625246 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.625250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.625985 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.625987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.626488 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.626495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.626916 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.626923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.626981 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.627423 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.627434 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.628688 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.630202 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.630205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.630207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.630391 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.630758 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.630762 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.630764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.630936 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.631314 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.631316 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.631414 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.631510 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.631635 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 22:21:43.631827 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.624717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.26.125.60:41171)
0811 22:21:43.624823 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.624826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.624881 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:21:43.624919 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.624921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.624957 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.625013 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.625325 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.625862 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.626578 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.626811 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.626940 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.628860 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.628863 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.628867 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.628890 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.630101 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.630504 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.630507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.630508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.630596 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.631034 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.631037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.631038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.631233 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.631310 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.631560 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.631700 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":39}
I20260811 22:21:43.635468 6661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Bootstrap starting.
I20260811 22:21:43.637815 6661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:43.638545 6661 log.cc:824] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:43.640558 6661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: No bootstrap required, opened a new log
I20260811 22:21:43.643723 6661 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } }
I20260811 22:21:43.643944 6661 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:43.643978 6661 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8150f1dce374c6c979f79e3220374ff, State: Initialized, Role: FOLLOWER
I20260811 22:21:43.644409 6661 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } }
I20260811 22:21:43.644933 6667 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } } }
I20260811 22:21:43.645067 6667 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:43.645684 6661 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:21:43.651100 6680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:43.651201 6680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:43.654971 6366 master_runner.cc:428] Detected that this master e8150f1dce374c6c979f79e3220374ff is joining an existing cluster
I20260811 22:21:43.655068 6366 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.125.60:41171
I20260811 22:21:43.666044 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 5485
I20260811 22:21:43.667543 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.658528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.26.125.60:41171)
0811 22:21:43.658609 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.658612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.658982 (+ 370us) server_negotiation.cc:407] Connection header received
0811 22:21:43.659105 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.659107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.659138 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.659178 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.659767 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.660526 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.661894 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.662220 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.662288 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.663931 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.663935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.663937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.663964 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.665722 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.666159 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.666163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.666165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.666260 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.666509 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.666511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.666513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.666735 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.666817 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.667217 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.667355 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 22:21:43.668208 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.658532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.0.0.1:42196)
0811 22:21:43.658917 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.658928 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.659026 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.659288 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.659291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.659511 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.659648 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.659657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.660646 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.660649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.661758 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.661772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.662503 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.662506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.662717 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.663396 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.663410 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.663795 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.665831 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.665834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.665836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.666079 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.666355 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.666358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.666359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.666435 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.666921 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.666925 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.667031 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.667825 (+ 794us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.668001 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":141,"spinlock_wait_cycles":1792,"thread_start_us":123,"threads_started":1}
I20260811 22:21:43.688437 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:34713
--local_ip_for_outbound_sockets=127.26.125.1
--tserver_master_addrs=127.26.125.62:45967
--webserver_port=41191
--webserver_interface=127.26.125.1
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab
--principal=oryx/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207,127.26.125.60:41171 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:43.696807 6694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.657734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48270)
0811 22:21:43.658652 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.658663 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.658801 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.659228 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.659231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.659456 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.659647 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.659657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.660763 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.660766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.661791 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.661799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.662503 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.662506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.662722 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.691945 (+ 29223us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.691961 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.692344 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.694476 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.694480 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.694483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.694916 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.695249 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.695252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.695254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.695382 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.696189 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.696192 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.696304 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.696409 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.696585 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":270,"spinlock_wait_cycles":9600,"thread_start_us":128,"threads_started":1}
I20260811 22:21:43.697150 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.658218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.26.125.61:36207)
0811 22:21:43.658375 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.658380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.658397 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:21:43.658634 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.658638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.658696 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.658785 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.659554 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.660474 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.662279 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.662631 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.662709 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.692316 (+ 29607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.692321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.692325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.692360 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.694312 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.695043 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.695047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.695049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.695144 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.695491 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.695495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.695496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.696051 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.696142 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.696836 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.697003 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:21:43.697341 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.657870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.26.125.62:45967)
0811 22:21:43.658114 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.658119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.658732 (+ 613us) server_negotiation.cc:407] Connection header received
0811 22:21:43.658981 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.658984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.659038 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.659089 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.659829 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.660599 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.661947 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.662285 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.662347 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.692575 (+ 30228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.692579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.692582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.692618 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.694313 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.695043 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.695047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.695049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.695144 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.695491 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.695495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.695496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.695752 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.695842 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.697079 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.697202 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260811 22:21:43.698194 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.657733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.0.0.1:54044)
0811 22:21:43.658266 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.658280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.658386 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.658960 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.658963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.659234 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.659403 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.659411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.660664 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.660667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.662139 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.662151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.662897 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.662901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.663000 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.691485 (+ 28485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.691508 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.692055 (+ 547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.694476 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.694480 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.694482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.694916 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.695252 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.695254 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.695256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.695382 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.696552 (+ 1170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.696555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.696647 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.697835 (+ 1188us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.698018 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":45,"spinlock_wait_cycles":6912,"thread_start_us":135,"threads_started":1}
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486902, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.62@KRBTEST.COM for oryx/127.26.125.60@KRBTEST.COM
I20260811 22:21:43.709177 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.701613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.26.125.60:41171)
0811 22:21:43.701723 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.701727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.701777 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:21:43.701952 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.701955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.702003 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.702067 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.702495 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.703070 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.703847 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.704075 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.704162 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.706222 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.706226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.706229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.706254 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.707578 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.707991 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.707995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.707997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.708078 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.708330 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.708334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.708336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.708540 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.708628 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.708889 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.709001 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:21:43.709177 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.701545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.0.0.1:42212)
0811 22:21:43.701722 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.701735 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.701822 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.702097 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.702099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.702288 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.702387 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.702393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.703179 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.703182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.703738 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.703746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.704180 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.704183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.704279 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.704799 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.704813 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.706078 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.707682 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.707686 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.707688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.707901 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.708162 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.708165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.708167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.708249 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.708631 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.708635 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.708735 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.708821 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.708996 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 22:21:43.710997 5374 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.125.60:41171: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e8150f1dce374c6c979f79e3220374ff"
member_type: NON_VOTER
last_known_addr {
host: "127.26.125.60"
port: 41171
}
attrs {
promote: true
}
}
dest_uuid: "bee3d139283d4c78966aa9c23c82c6ca"
cas_config_opid_index: 5
I20260811 22:21:43.711678 5374 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } }
I20260811 22:21:43.712844 6356 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e8150f1dce374c6c979f79e3220374ff (127.26.125.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.713662 6609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:43.714313 6356 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.125.60:41171
I20260811 22:21:43.714507 6356 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.714607 6356 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:43.715075 6366 master.cc:537] Master@127.26.125.60:41171 shutting down...
I20260811 22:21:43.715322 6609 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Refusing update from remote peer bee3d139283d4c78966aa9c23c82c6ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:21:43.715991 6356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:21:43.718894 5347 consensus_peers.cc:487] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca -> Peer e8150f1dce374c6c979f79e3220374ff (127.26.125.60:41171): Couldn't send request to peer e8150f1dce374c6c979f79e3220374ff. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:21:43.722339 6366 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:43.722781 6366 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:43.723084 6366 tablet_replica.cc:330] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: stopping tablet replica
I20260811 22:21:43.723940 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.713055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.0.0.1:54060)
0811 22:21:43.718383 (+ 5328us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.718403 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.718506 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.719289 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.719292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.719314 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:43.719421 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.719429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.720579 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.720583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.722119 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.722135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.723420 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.723425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.723527 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.723586 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.723674 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.723769 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5238}
I20260811 22:21:43.724475 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.713346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54060 (local address 127.26.125.61:36207)
0811 22:21:43.713592 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.713598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.718611 (+ 5013us) server_negotiation.cc:407] Connection header received
0811 22:21:43.718681 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.718684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.718763 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.719111 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:43.719573 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.720332 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.722298 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.723023 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.723148 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.723341 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.724214 (+ 873us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.724334 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260811 22:21:43.726189 6465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 LEARNER]: Refusing update from remote peer bee3d139283d4c78966aa9c23c82c6ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:21:43.726980 6356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 22:21:43.729444 6465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e8150f1dce374c6c979f79e3220374ff (127.26.125.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.732241 6700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.732394 6700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:21:43.732563 6516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee3d139283d4c78966aa9c23c82c6ca. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: NON_VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: true } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.732692 6516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:21:43.732769 6357 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: attempting to promote NON_VOTER 2587ac2153bd4418bbb8b86a8184cde2 to VOTER
I20260811 22:21:43.733237 6357 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } }
I20260811 22:21:43.734607 6465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 LEARNER]: Refusing update from remote peer bee3d139283d4c78966aa9c23c82c6ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 22:21:43.734706 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.734361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.0.0.1:42220)
0811 22:21:43.734525 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.734612 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:21:43.735237 6356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 22:21:43.735565 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.735300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.0.0.1:42222)
0811 22:21:43.735442 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.735496 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 22:21:43.735688 5347 consensus_peers.cc:487] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca -> Peer e8150f1dce374c6c979f79e3220374ff (127.26.125.60:41171): Couldn't send request to peer e8150f1dce374c6c979f79e3220374ff. Status: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:21:43.737393 6361 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2587ac2153bd4418bbb8b86a8184cde2 (127.26.125.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.738030 6465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2587ac2153bd4418bbb8b86a8184cde2 (127.26.125.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.739142 6361 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.739259 6361 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:43.739427 6361 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.739517 6361 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:43.739584 6516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.739704 6516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:43.742362 6366 master.cc:552] Master@127.26.125.60:41171 shutdown complete.
I20260811 22:21:43.742416 6366 master_runner.cc:305] Clearing existing system tablet
I20260811 22:21:43.742728 6366 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:43.742789 6366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:43.743984 6366 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:43.744800 6702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.745040 6366 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:43.745172 6366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:43.745210 6366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
uuid: "e8150f1dce374c6c979f79e3220374ff"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "7d4a62370dd50d5ec8d7d5e12bef8d45"
server_key_iv: "58d260c4fc29b5cd58dc7ff7ed02acbb"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.747263 6366 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:43.749401 6366 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:21:43.749511 6366 log.cc:1196] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 22:21:43.749744 6366 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Deleting consensus metadata
I20260811 22:21:43.749912 6366 master_runner.cc:315] Copying system tablet from 127.26.125.62:45967
I20260811 22:21:43.751075 6366 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: tablet copy: Beginning tablet copy session from remote peer at address 127.26.125.62:45967
I20260811 22:21:43.762068 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.754985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48286 (local address 127.26.125.62:45967)
0811 22:21:43.755149 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.755153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.755294 (+ 141us) server_negotiation.cc:407] Connection header received
0811 22:21:43.755472 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.755476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.755549 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.755625 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.756091 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.756743 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.757549 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.757769 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.757857 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.758982 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.758986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.758989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.759015 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.760434 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.760833 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.760837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.760839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.760927 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.761178 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.761182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.761184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.761375 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.761471 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.761790 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.761903 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:21:43.762081 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.754779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48286)
0811 22:21:43.755204 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.755218 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.755341 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.755645 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.755648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.755850 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.755974 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.755983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.756860 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.756864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.757440 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.757449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.757873 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.757877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.757983 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.758478 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.758492 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.758826 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.760532 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.760535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.760538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.760746 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.761009 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.761012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.761015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.761100 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.761471 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.761475 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.761627 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.761719 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.761884 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":153,"threads_started":1}
I20260811 22:21:43.762903 5405 tablet_copy_service.cc:140] P bee3d139283d4c78966aa9c23c82c6ca: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e8150f1dce374c6c979f79e3220374ff ({username='oryx', principal='oryx/127.26.125.60@KRBTEST.COM'} at 127.0.0.1:48286)
I20260811 22:21:43.763008 5405 tablet_copy_service.cc:161] P bee3d139283d4c78966aa9c23c82c6ca: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e8150f1dce374c6c979f79e3220374ff at {username='oryx', principal='oryx/127.26.125.60@KRBTEST.COM'} at 127.0.0.1:48286: session id = e8150f1dce374c6c979f79e3220374ff-00000000000000000000000000000000
I20260811 22:21:43.764017 5405 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:21:43.765214 6366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:21:43.767633 6366 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: tablet copy: Starting download of 0 data blocks...
I20260811 22:21:43.767840 6366 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: tablet copy: Starting download of 1 WAL segments...
I20260811 22:21:43.769589 6366 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:21:43.771562 5405 tablet_copy_service.cc:342] P bee3d139283d4c78966aa9c23c82c6ca: Request end of tablet copy session e8150f1dce374c6c979f79e3220374ff-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.125.60@KRBTEST.COM'} at 127.0.0.1:48286
I20260811 22:21:43.771651 5405 tablet_copy_service.cc:433] P bee3d139283d4c78966aa9c23c82c6ca: ending tablet copy session e8150f1dce374c6c979f79e3220374ff-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e8150f1dce374c6c979f79e3220374ff
I20260811 22:21:43.778415 6366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:43.780192 6714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:43.780422 6366 server_base.cc:1034] running on GCE node
W20260811 22:21:43.780288 6718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:43.786130 6715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:43.786504 6366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:43.786942 6366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:43.788125 6366 hybrid_clock.cc:648] HybridClock initialized: now 1786486903788114 us; error 39 us; skew 500 ppm
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:43.789673 6366 init.cc:377] Logged in from keytab as oryx/127.26.125.60@KRBTEST.COM (short username oryx)
I20260811 22:21:43.790194 6366 webserver.cc:460] Webserver started at http://127.26.125.60:33849/ using document root <none> and password file <none>
I20260811 22:21:43.790390 6366 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:43.790434 6366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:43.790678 5936 heartbeater.cc:503] Master 127.26.125.62:45967 was elected leader, sending a full tablet report...
I20260811 22:21:43.791769 6366 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:43.792785 6735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.792992 6366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:43.793056 6366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
uuid: "e8150f1dce374c6c979f79e3220374ff"
format_stamp: "Formatted at 2026-08-11 22:21:43 on dist-test-slave-5bn2"
server_key: "7d4a62370dd50d5ec8d7d5e12bef8d45"
server_key_iv: "58d260c4fc29b5cd58dc7ff7ed02acbb"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.793133 6366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:43.808643 6366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:43.810645 6366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:43.815560 6366 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.60:41171
I20260811 22:21:43.815580 6836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.60:41171 every 8 connection(s)
I20260811 22:21:43.816286 6366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
I20260811 22:21:43.818064 6837 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:21:43.818995 6837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Bootstrap starting.
I20260811 22:21:43.828776 6837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:21:43.829077 6837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Bootstrap complete.
I20260811 22:21:43.829666 6837 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } }
I20260811 22:21:43.829797 6837 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e8150f1dce374c6c979f79e3220374ff, State: Initialized, Role: LEARNER
I20260811 22:21:43.829910 6837 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } }
I20260811 22:21:43.830282 6842 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:43.830461 6837 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:43.830474 6842 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: This master's current role is: LEARNER
I20260811 22:21:43.833884 6858 catalog_manager.cc:1259] Loaded cluster ID: 357565593f734e1ca5de1813dad5b7a2
I20260811 22:21:43.833945 6858 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: loading cluster ID for follower catalog manager: success
I20260811 22:21:43.835649 6858 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: acquiring CA information for follower catalog manager: success
I20260811 22:21:43.836889 6858 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:21:43.862142 6696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:43.862427 6696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:43.862473 6696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:43.866580 6696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:43.866662 6696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:43.866722 6696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:43.872380 6696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab
--principal=oryx/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:34713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=41191
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207,127.26.125.60:41171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:43.873507 6696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:43.874564 6696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:43.885236 6867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:43.885277 6865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:43.885303 6863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:43.885802 6696 server_base.cc:1034] running on GCE node
I20260811 22:21:43.886304 6696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:43.887063 6696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:43.888224 6696 hybrid_clock.cc:648] HybridClock initialized: now 1786486903888189 us; error 49 us; skew 500 ppm
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:43.891184 6696 init.cc:377] Logged in from keytab as oryx/127.26.125.1@KRBTEST.COM (short username oryx)
I20260811 22:21:43.892170 6696 webserver.cc:460] Webserver started at http://127.26.125.1:41191/ using document root <none> and password file <none>
I20260811 22:21:43.892755 6696 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:43.892835 6696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:43.897135 6696 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:21:43.899586 6876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.901023 6696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:43.901144 6696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "bde0b8d5e9a74062ba71967366d77a28"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "ab68ad5c9172dc3fd2541aeb809ebebc"
server_key_iv: "3bbbae982c93b51846ea4318817ae2d7"
server_key_version: "encryptionkey@0"
I20260811 22:21:43.901741 6696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:43.935952 6696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:43.939312 6696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:43.939499 6696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:43.940147 6696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:43.941056 6696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:43.941104 6696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.941148 6696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:43.941177 6696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:43.950547 6696 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:34713
I20260811 22:21:43.950577 7073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:34713 every 8 connection(s)
I20260811 22:21:43.951666 6696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:43.958652 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 6696
I20260811 22:21:43.958878 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 5707
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.60@KRBTEST.COM
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:43.969208 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.953619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:33093 (local address 127.26.125.62:45967)
0811 22:21:43.953752 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.953756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.955413 (+ 1657us) server_negotiation.cc:407] Connection header received
0811 22:21:43.956389 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.956392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.956504 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.956587 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.958792 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.959558 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.962479 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.962949 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.963040 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.965400 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.965404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.965407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.965435 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.966834 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.967485 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.967488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.967490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.967569 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.967896 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.967909 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.967910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.968124 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.968196 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.968931 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.969055 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:21:43.969602 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.953619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:34679 (local address 127.26.125.60:41171)
0811 22:21:43.953891 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.953895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.955189 (+ 1294us) server_negotiation.cc:407] Connection header received
0811 22:21:43.956275 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.956279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.956341 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.956430 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.957823 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.958906 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.961227 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.961475 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.961552 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.965215 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.965220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.965224 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.965259 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.967431 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.968018 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.968023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.968025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.968135 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.968491 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.968495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.968497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.968762 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.968841 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.969258 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.969411 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
I20260811 22:21:43.971458 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.954145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.1:33093)
0811 22:21:43.955311 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.955323 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.956041 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.956642 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.956651 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.957208 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.957687 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.957697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.959694 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.959697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.960880 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.960889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.963259 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.963263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.963372 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.963962 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.963979 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.965264 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.966991 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.966997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.967016 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.967390 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.967653 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.967655 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.967657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.967818 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.968294 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.968300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.968672 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.969439 (+ 767us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.970352 (+ 913us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"mutex_wait_us":193,"spinlock_wait_cycles":27776,"thread_start_us":107,"threads_started":1}
I20260811 22:21:43.971463 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.954145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.26.125.1:34679)
0811 22:21:43.955106 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.955118 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.956057 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.956641 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.956650 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.957148 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.957673 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.957688 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.959068 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.959071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.961136 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.961141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.961768 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.961772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.962052 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.962805 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.962826 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.964869 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.967590 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.967593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.967596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.967874 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.968268 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.968272 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.968274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.968371 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.968921 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.968924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.969046 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.970286 (+ 1240us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.970461 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":646,"spinlock_wait_cycles":10112,"thread_start_us":149,"threads_started":1}
I20260811 22:21:43.980741 7076 heartbeater.cc:347] Connected to a master server at 127.26.125.60:41171
I20260811 22:21:43.981101 7076 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:43.981594 7077 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45967
I20260811 22:21:43.981693 7077 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:43.981796 7076 heartbeater.cc:511] Master 127.26.125.60:41171 requested a full tablet report, sending...
I20260811 22:21:43.981884 7077 heartbeater.cc:511] Master 127.26.125.62:45967 requested a full tablet report, sending...
I20260811 22:21:43.983348 5374 ts_manager.cc:194] Re-registered known tserver with Master: bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713)
I20260811 22:21:43.983922 6763 ts_manager.cc:194] Registered new tserver with Master: bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713)
I20260811 22:21:43.984131 5374 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:33093
Aug 11 22:21:43 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.61@KRBTEST.COM
I20260811 22:21:43.987203 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:34859
--local_ip_for_outbound_sockets=127.26.125.2
--tserver_master_addrs=127.26.125.62:45967
--webserver_port=37373
--webserver_interface=127.26.125.2
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab
--principal=oryx/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207,127.26.125.60:41171 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:43.988343 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.954145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.26.125.1:35895)
0811 22:21:43.954770 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:43.954833 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:43.956035 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:43.956642 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:43.956651 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:43.957215 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:43.957689 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.957697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.979634 (+ 21937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.979638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.980337 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:43.980353 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.981174 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.981178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:43.981281 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.982047 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:43.982062 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:43.984772 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:43.986738 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.986742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.986744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.987010 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.987341 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:43.987345 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:43.987347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:43.987452 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:43.987886 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:43.987888 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:43.987971 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 22:21:43.988061 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:21:43.988196 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":97,"spinlock_wait_cycles":14976,"thread_start_us":665,"threads_started":1}
I20260811 22:21:43.988442 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:43.954118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:35895 (local address 127.26.125.61:36207)
0811 22:21:43.954237 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:43.954241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:43.954971 (+ 730us) server_negotiation.cc:407] Connection header received
0811 22:21:43.956286 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:43.956289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:43.956341 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:43.956397 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:43.957861 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.958663 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.980588 (+ 21925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:43.980883 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:43.980978 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:43.984972 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:43.984977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:43.984980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:43.985013 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:43.986627 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.987125 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.987129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.987131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.987238 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:43.987560 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:43.987563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:43.987565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:43.987793 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:43.987902 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:43.988147 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 22:21:43.988277 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:21:43.989156 7079 heartbeater.cc:347] Connected to a master server at 127.26.125.61:36207
I20260811 22:21:43.989271 7079 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:43.989516 7079 heartbeater.cc:511] Master 127.26.125.61:36207 requested a full tablet report, sending...
I20260811 22:21:43.990579 6445 ts_manager.cc:194] Registered new tserver with Master: bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713)
I20260811 22:21:44.037990 6159 heartbeater.cc:503] Master 127.26.125.62:45967 was elected leader, sending a full tablet report...
W20260811 22:21:44.111753 7091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:44.112042 7091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:44.112087 7091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:44.116446 7091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:44.116515 7091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:44.116596 7091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:44.122270 7091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab
--principal=oryx/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:34859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=37373
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207,127.26.125.60:41171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:44.123503 7091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:44.124541 7091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:44.134757 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:44.134757 7101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:44.134802 7091 server_base.cc:1034] running on GCE node
W20260811 22:21:44.134758 7109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:44.135521 7091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:44.136212 7091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:44.137431 7091 hybrid_clock.cc:648] HybridClock initialized: now 1786486904137402 us; error 42 us; skew 500 ppm
Aug 11 22:21:44 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:44.140506 7091 init.cc:377] Logged in from keytab as oryx/127.26.125.2@KRBTEST.COM (short username oryx)
I20260811 22:21:44.141477 7091 webserver.cc:460] Webserver started at http://127.26.125.2:37373/ using document root <none> and password file <none>
I20260811 22:21:44.142024 7091 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:44.142096 7091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:44.146062 7091 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:44.148306 7123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:44.149652 7091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:21:44.149775 7091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "13fbdc730e1446b8937ff734023d6305"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "8bdd5b863b234185359223cb02f402ec"
server_key_iv: "a700cb6d0e38d6eb8012753cc02c6048"
server_key_version: "encryptionkey@0"
I20260811 22:21:44.150337 7091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:44.170648 7091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:44.173600 7091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:44.173789 7091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:44.174443 7091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:44.175408 7091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:44.175458 7091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:44.175501 7091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:44.175529 7091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:44.184530 7091 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:34859
I20260811 22:21:44.184594 7318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:34859 every 8 connection(s)
I20260811 22:21:44.185626 7091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:44.187583 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 7091
I20260811 22:21:44.187781 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 5942
I20260811 22:21:44.199254 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:45661
--local_ip_for_outbound_sockets=127.26.125.3
--tserver_master_addrs=127.26.125.62:45967
--webserver_port=45109
--webserver_interface=127.26.125.3
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab
--principal=oryx/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207,127.26.125.60:41171 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:21:44 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.60@KRBTEST.COM
Aug 11 22:21:44 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
Aug 11 22:21:44 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.61@KRBTEST.COM
I20260811 22:21:44.207556 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.187811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:36961 (local address 127.26.125.60:41171)
0811 22:21:44.188062 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.188066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.192322 (+ 4256us) server_negotiation.cc:407] Connection header received
0811 22:21:44.193157 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.193161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.193225 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.193274 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.195401 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.196235 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198166 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.198526 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198605 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.203092 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.203096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.203098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.203131 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.205077 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.205619 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.205623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.205625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.205708 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.206071 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.206074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.206076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.206313 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.206399 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.207212 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.207365 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260811 22:21:44.207615 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.187672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:35011 (local address 127.26.125.62:45967)
0811 22:21:44.187965 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.187969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.191981 (+ 4012us) server_negotiation.cc:407] Connection header received
0811 22:21:44.193156 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.193159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.193216 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.193268 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.195397 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.196224 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198148 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.198501 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198575 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.202768 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.202772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.202775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.202812 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.204953 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.205558 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.205562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.205564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.205661 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.206092 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.206096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.206098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.206338 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.206433 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.207221 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.207360 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260811 22:21:44.208702 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.188119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.2:35011)
0811 22:21:44.191790 (+ 3671us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.191849 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.192899 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.193699 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.193709 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.194343 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:44.195226 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.195248 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.196419 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.196422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.198009 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.198020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.198808 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.199063 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.199945 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:44.199958 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:44.202545 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:44.205081 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.205084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.205086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.205435 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.205774 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.205777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.205779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.205974 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.206494 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:44.206497 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:44.206905 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.208159 (+ 1254us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.208312 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3451,"mutex_wait_us":324,"spinlock_wait_cycles":88960,"thread_start_us":2355,"threads_started":1}
I20260811 22:21:44.208707 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.188119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.26.125.2:33319)
0811 22:21:44.192505 (+ 4386us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.192518 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.192943 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.193699 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.193709 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.194367 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:44.195228 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.195248 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.196411 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.196414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.198009 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.198020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.198961 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.199058 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.199887 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:44.199939 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:44.202901 (+ 2962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:44.204910 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.204939 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.204969 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.205433 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.205754 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.205757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.205759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.205967 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.206532 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:44.206535 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:44.206901 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.207130 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.207505 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4287,"mutex_wait_us":3624,"spinlock_wait_cycles":35072,"thread_start_us":122,"threads_started":1}
I20260811 22:21:44.209100 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.191293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:33319 (local address 127.26.125.61:36207)
0811 22:21:44.191522 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.191527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.192602 (+ 1075us) server_negotiation.cc:407] Connection header received
0811 22:21:44.193154 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.193158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.193216 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.193268 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.195397 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.196224 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198148 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.198501 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198574 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.203200 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.203205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.203207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.203239 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.204767 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.205553 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.205557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.205559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.205652 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.206071 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.206074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.206076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.206362 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.206462 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.208846 (+ 2384us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.208975 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260811 22:21:44.208707 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.188305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.26.125.2:36961)
0811 22:21:44.192181 (+ 3876us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.192196 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.192910 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.193700 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.193709 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.194441 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:44.195227 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.195248 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.196414 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.196417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.198008 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.198020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.198808 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.198812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.199721 (+ 909us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.200091 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:44.200106 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:44.202901 (+ 2795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:44.205195 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.205198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.205201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.205514 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.205805 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.205807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.205809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.205962 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.206483 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:44.206491 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:44.206901 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.207784 (+ 883us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.207927 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3807,"mutex_wait_us":212,"spinlock_wait_cycles":47872,"thread_start_us":2245,"threads_started":1}
I20260811 22:21:44.210484 7320 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45967
I20260811 22:21:44.210508 7321 heartbeater.cc:347] Connected to a master server at 127.26.125.61:36207
I20260811 22:21:44.210714 7320 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:44.210721 7321 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:44.211355 7320 heartbeater.cc:511] Master 127.26.125.62:45967 requested a full tablet report, sending...
I20260811 22:21:44.211359 7321 heartbeater.cc:511] Master 127.26.125.61:36207 requested a full tablet report, sending...
I20260811 22:21:44.211386 7319 heartbeater.cc:347] Connected to a master server at 127.26.125.60:41171
I20260811 22:21:44.211540 7319 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:44.211683 7319 heartbeater.cc:511] Master 127.26.125.60:41171 requested a full tablet report, sending...
I20260811 22:21:44.212661 5374 ts_manager.cc:194] Re-registered known tserver with Master: 13fbdc730e1446b8937ff734023d6305 (127.26.125.2:34859)
I20260811 22:21:44.212734 6763 ts_manager.cc:194] Registered new tserver with Master: 13fbdc730e1446b8937ff734023d6305 (127.26.125.2:34859)
I20260811 22:21:44.213105 6445 ts_manager.cc:194] Registered new tserver with Master: 13fbdc730e1446b8937ff734023d6305 (127.26.125.2:34859)
I20260811 22:21:44.213414 5374 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:35011
I20260811 22:21:44.265969 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.262415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.26.125.60:41171)
0811 22:21:44.262622 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.262628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.262727 (+ 99us) server_negotiation.cc:407] Connection header received
0811 22:21:44.262920 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.262922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.262982 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.263074 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:44.263344 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.263954 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.264975 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.265422 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.265538 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.265718 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.265755 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.265824 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260811 22:21:44.265969 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.262263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.0.0.1:42226)
0811 22:21:44.262585 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.262601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.262781 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.263107 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.263110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.263131 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:44.263235 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.263240 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.264065 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.264068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.264842 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.264852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.265550 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.265553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.265646 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.265670 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.265734 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.265819 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":88,"threads_started":1}
I20260811 22:21:44.266763 6782 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 22:21:44.267181 6842 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee3d139283d4c78966aa9c23c82c6ca. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:44.267354 6842 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: This master's current role is: LEARNER
W20260811 22:21:44.325169 7328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:44.325487 7328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:44.325552 7328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:44.329756 7328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:44.329833 7328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:44.329918 7328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:44.335785 7328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab
--principal=oryx/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:45661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=45109
--tserver_master_addrs=127.26.125.62:45967,127.26.125.61:36207,127.26.125.60:41171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:44.336948 7328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:44.338136 7328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:44.348348 7344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:44.348393 7328 server_base.cc:1034] running on GCE node
W20260811 22:21:44.348348 7337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:44.348384 7339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:44.349057 7328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:44.349762 7328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:44.350986 7328 hybrid_clock.cc:648] HybridClock initialized: now 1786486904350955 us; error 43 us; skew 500 ppm
Aug 11 22:21:44 dist-test-slave-5bn2 krb5kdc[5286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:44.354221 7328 init.cc:377] Logged in from keytab as oryx/127.26.125.3@KRBTEST.COM (short username oryx)
I20260811 22:21:44.355458 7328 webserver.cc:460] Webserver started at http://127.26.125.3:45109/ using document root <none> and password file <none>
I20260811 22:21:44.356108 7328 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:44.356196 7328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:44.360630 7328 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:21:44.363144 7355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:44.364570 7328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:44.364693 7328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "c84c5ba6d5184657bd4a9c17a5765a80"
format_stamp: "Formatted at 2026-08-11 22:21:42 on dist-test-slave-5bn2"
server_key: "81448ec96f13417ca9346329c4f68595"
server_key_iv: "747718430291913484ddbea71f49498b"
server_key_version: "encryptionkey@0"
I20260811 22:21:44.365274 7328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:44.385697 7328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:44.389362 7328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:44.389565 7328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:44.390236 7328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:44.391202 7328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:44.391252 7328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:44.391299 7328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:44.391327 7328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:44.400481 7328 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:45661
I20260811 22:21:44.400544 7542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:45661 every 8 connection(s)
I20260811 22:21:44.401580 7328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:44.408811 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 7328
Aug 11 22:21:44 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.61@KRBTEST.COM
Aug 11 22:21:44 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.60@KRBTEST.COM
Aug 11 22:21:44 dist-test-slave-5bn2 krb5kdc[5286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:44.419121 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.403590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:53949 (local address 127.26.125.60:41171)
0811 22:21:44.403720 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.403724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.405222 (+ 1498us) server_negotiation.cc:407] Connection header received
0811 22:21:44.406163 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.406166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.406235 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.406314 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.407914 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.408734 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.411011 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.411333 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.411407 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.414022 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.414027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.414029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.414059 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.416288 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.417082 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.417087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.417089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.417194 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.417641 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.417645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.417647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.417880 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.417953 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.418783 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.418950 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:21:44.420837 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.404282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:51403 (local address 127.26.125.61:36207)
0811 22:21:44.404968 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.404972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.406409 (+ 1437us) server_negotiation.cc:407] Connection header received
0811 22:21:44.406543 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.406546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.406598 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.406682 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.407899 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.408553 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.410512 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.410810 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.410922 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.413797 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.413802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.413805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.413834 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.417116 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.417608 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.417611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.417613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.417707 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.417977 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.417980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.417982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.418199 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.418282 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.420555 (+ 2273us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.420678 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591}
I20260811 22:21:44.420966 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.403944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.26.125.3:53949)
0811 22:21:44.405119 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.405131 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.405920 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.406499 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.406508 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.407174 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:44.407740 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.407754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.408875 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.408878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.410359 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.410370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.411560 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.411563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.411650 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.412103 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:44.412117 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:44.413859 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:44.416492 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.416500 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.416522 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.416959 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.417307 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.417310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.417312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.417529 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.418038 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:44.418045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:44.418391 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.419267 (+ 876us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.419490 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"mutex_wait_us":296,"spinlock_wait_cycles":56832,"thread_start_us":183,"threads_started":1}
I20260811 22:21:44.421113 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.404161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.26.125.3:51403)
0811 22:21:44.406322 (+ 2161us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.406335 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.406437 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.406711 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.406714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.407180 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:44.407740 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.407754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.408739 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.408742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.410350 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.410362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.411091 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.411094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.411181 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.411766 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:44.411781 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:44.413596 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:44.417231 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.417234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.417237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.417491 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.417802 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.417805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.417806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.417888 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.418324 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:44.418328 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:44.418419 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.418714 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.419421 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2100,"mutex_wait_us":275,"spinlock_wait_cycles":23040,"thread_start_us":114,"threads_started":1}
I20260811 22:21:44.422406 7543 heartbeater.cc:347] Connected to a master server at 127.26.125.60:41171
I20260811 22:21:44.422691 7543 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:44.423367 7543 heartbeater.cc:511] Master 127.26.125.60:41171 requested a full tablet report, sending...
I20260811 22:21:44.423995 7548 heartbeater.cc:347] Connected to a master server at 127.26.125.61:36207
I20260811 22:21:44.424028 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.413019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.26.125.61:36207)
0811 22:21:44.413644 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.413680 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.415067 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 22:21:44.415113 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.415115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.415169 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.415247 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.416227 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.417006 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.418106 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.418424 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.418644 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.420303 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.420307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.420310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.420335 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.421945 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.422314 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.422318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.422319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.422411 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.422712 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.422715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.422717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.422987 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.423110 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.423729 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.423857 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":85,"threads_started":1}
I20260811 22:21:44.424077 7548 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:44.424240 7548 heartbeater.cc:511] Master 127.26.125.61:36207 requested a full tablet report, sending...
I20260811 22:21:44.424324 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.403517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:34471 (local address 127.26.125.62:45967)
0811 22:21:44.403660 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.403665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.405191 (+ 1526us) server_negotiation.cc:407] Connection header received
0811 22:21:44.406131 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.406135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.406197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.406265 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.407878 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.408734 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.410471 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.410693 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.410781 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.414830 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.414834 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.414837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.414913 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.416590 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.422909 (+ 6319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.422913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.422914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.423017 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.423351 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.423355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.423356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.423598 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.423683 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.424046 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.424165 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:21:44.424772 6763 ts_manager.cc:194] Registered new tserver with Master: c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661)
I20260811 22:21:44.424962 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.403944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.3:34471)
0811 22:21:44.405054 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.405096 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.405930 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.406501 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.406509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.407203 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:44.407737 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.407751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.409073 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.409075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.410360 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.410370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.410837 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.410841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.411071 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.411672 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:44.411696 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:44.414661 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:44.422507 (+ 7846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.422511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.422513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.422762 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.423143 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:44.423146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:44.423148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:44.423242 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:44.423747 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:44.423750 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:44.423847 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.424633 (+ 786us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.424787 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":130,"spinlock_wait_cycles":13568,"thread_start_us":702,"threads_started":1}
I20260811 22:21:44.425252 6445 ts_manager.cc:194] Registered new tserver with Master: c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661)
I20260811 22:21:44.425869 7545 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45967
I20260811 22:21:44.425959 7545 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:44.426110 7545 heartbeater.cc:511] Master 127.26.125.62:45967 requested a full tablet report, sending...
I20260811 22:21:44.426693 5374 ts_manager.cc:194] Re-registered known tserver with Master: c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661)
I20260811 22:21:44.427424 5374 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:34471
I20260811 22:21:44.429214 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.414890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48298 (local address 127.26.125.62:45967)
0811 22:21:44.415923 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.415940 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.415966 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:21:44.416008 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.416011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.416059 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.417107 (+ 1048us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.417283 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.418045 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.419163 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.419821 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.420289 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.421887 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.421890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.421893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.421922 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.426419 (+ 4497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.426909 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.426913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.426914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.427011 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.428171 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.428173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.428175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.428343 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.428468 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.428756 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.428881 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":951,"thread_start_us":112,"threads_started":1}
I20260811 22:21:44.433521 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.411236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.26.125.60:41171)
0811 22:21:44.412662 (+ 1426us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.412667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.425331 (+ 12664us) server_negotiation.cc:407] Connection header received
0811 22:21:44.425610 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.425614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.425669 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.425755 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.426366 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.427099 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.428156 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.428482 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.428668 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.430356 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.430360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.430362 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.430394 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.431786 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.432299 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.432303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.432305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.432405 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.432661 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.432664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.432666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.432865 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.432989 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.433205 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.433335 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1328,"thread_start_us":964,"threads_started":1}
I20260811 22:21:44.432224 5374 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:44.435593 5374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:44.438150 6357 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: NON_VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: true } } }
I20260811 22:21:44.438287 6357 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:44.442201 6357 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: attempting to promote NON_VOTER e8150f1dce374c6c979f79e3220374ff to VOTER
I20260811 22:21:44.442561 6357 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } }
I20260811 22:21:44.443928 6465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Refusing update from remote peer bee3d139283d4c78966aa9c23c82c6ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:21:44.444285 7570 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:21:44.444344 6781 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 LEARNER]: Refusing update from remote peer bee3d139283d4c78966aa9c23c82c6ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:21:44.444762 7570 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:21:44.445993 6357 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e8150f1dce374c6c979f79e3220374ff (127.26.125.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:44.446534 6465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e8150f1dce374c6c979f79e3220374ff (127.26.125.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:44.447322 6781 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e8150f1dce374c6c979f79e3220374ff (127.26.125.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:44.447657 6357 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:44.447744 7569 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:44.447841 7569 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:44.447928 7569 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:44.447993 7569 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:44.447749 6357 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:44.452236 6842 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:44.452353 6842 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:44.453356 7583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bee3d139283d4c78966aa9c23c82c6ca" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:44.453516 7583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:44.456712 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.450567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:34713 (local address 127.0.0.1:34438)
0811 22:21:44.451223 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.451238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.451321 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.452023 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.452026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.452047 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:44.452155 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.452161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.453759 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.453763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.454826 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.454837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.456318 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.456322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.456417 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.456445 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.456489 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.456581 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586}
I20260811 22:21:44.457388 7587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.451002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.26.125.1:34713)
0811 22:21:44.451323 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.451343 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.451374 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:21:44.451619 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.451626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.451858 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.452301 (+ 443us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:44.452427 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.453619 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.455300 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.456182 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.456970 (+ 788us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.457144 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.457192 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.457269 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":98,"threads_started":1}
I20260811 22:21:44.460286 6956 tablet_service.cc:1467] Processing CreateTablet for tablet 13d3afcaa18941f4a4843fedf1dee3fb (DEFAULT_TABLE table=test-table [id=ab709ca9160342b9b3a27222b71c7b89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:44.461509 6956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d3afcaa18941f4a4843fedf1dee3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:44.464062 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.454640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:34859 (local address 127.0.0.1:60590)
0811 22:21:44.454984 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.454997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.455087 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.459572 (+ 4485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.459575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.459596 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:44.459685 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.459691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.461053 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.461056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.462259 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.462272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.463632 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.463636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.463725 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.463754 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.463834 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.463908 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":110,"threads_started":1}
I20260811 22:21:44.464133 7591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.455098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.26.125.2:34859)
0811 22:21:44.459098 (+ 4000us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.459110 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.459136 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:21:44.459216 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.459221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.459418 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.459579 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:44.459796 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.460939 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.462448 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.463495 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.463623 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.463780 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.463865 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.463966 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3889,"thread_start_us":95,"threads_started":1}
I20260811 22:21:44.467420 7208 tablet_service.cc:1467] Processing CreateTablet for tablet 13d3afcaa18941f4a4843fedf1dee3fb (DEFAULT_TABLE table=test-table [id=ab709ca9160342b9b3a27222b71c7b89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:44.468575 7208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d3afcaa18941f4a4843fedf1dee3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:44.471619 7595 tablet_bootstrap.cc:492] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Bootstrap starting.
I20260811 22:21:44.472496 7588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.451343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:45661 (local address 127.0.0.1:39294)
0811 22:21:44.452991 (+ 1648us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.453005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.453101 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.455685 (+ 2584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.455688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.455707 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:44.455807 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.455814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.457138 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.457141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.458342 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.458353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.472027 (+ 13674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.472031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.472140 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.472173 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.472232 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.472327 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1579,"thread_start_us":107,"threads_started":1}
I20260811 22:21:44.473296 7589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.454152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39294 (local address 127.26.125.3:45661)
0811 22:21:44.455001 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.455027 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.455053 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:21:44.455132 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.455199 (+ 67us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.455528 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.455722 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:44.455934 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.457003 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.470940 (+ 13937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.471853 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.472804 (+ 951us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.472992 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.473052 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.473150 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":137,"threads_started":1}
I20260811 22:21:44.474363 7595 tablet_bootstrap.cc:654] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:44.475195 7595 log.cc:824] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:44.476992 7597 tablet_bootstrap.cc:492] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Bootstrap starting.
I20260811 22:21:44.476636 7436 tablet_service.cc:1467] Processing CreateTablet for tablet 13d3afcaa18941f4a4843fedf1dee3fb (DEFAULT_TABLE table=test-table [id=ab709ca9160342b9b3a27222b71c7b89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:44.477563 7595 tablet_bootstrap.cc:492] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: No bootstrap required, opened a new log
I20260811 22:21:44.477763 7595 ts_tablet_manager.cc:1397] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:21:44.477809 7436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d3afcaa18941f4a4843fedf1dee3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:44.479723 7597 tablet_bootstrap.cc:654] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:44.480680 7597 log.cc:824] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:44.481797 7595 raft_consensus.cc:348] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.482084 7595 raft_consensus.cc:374] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:44.482146 7595 raft_consensus.cc:729] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde0b8d5e9a74062ba71967366d77a28, State: Initialized, Role: FOLLOWER
I20260811 22:21:44.482641 7595 consensus_queue.cc:260] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.483489 7597 tablet_bootstrap.cc:492] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: No bootstrap required, opened a new log
I20260811 22:21:44.483700 7597 ts_tablet_manager.cc:1397] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:21:44.483775 7595 ts_tablet_manager.cc:1428] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:21:44.484122 7077 heartbeater.cc:503] Master 127.26.125.62:45967 was elected leader, sending a full tablet report...
I20260811 22:21:44.488268 7600 tablet_bootstrap.cc:492] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Bootstrap starting.
I20260811 22:21:44.490010 7597 raft_consensus.cc:348] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.490257 7597 raft_consensus.cc:374] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:44.490293 7597 raft_consensus.cc:729] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fbdc730e1446b8937ff734023d6305, State: Initialized, Role: FOLLOWER
I20260811 22:21:44.490620 7600 tablet_bootstrap.cc:654] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:44.490816 7597 consensus_queue.cc:260] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.491549 7320 heartbeater.cc:503] Master 127.26.125.62:45967 was elected leader, sending a full tablet report...
I20260811 22:21:44.491533 7600 log.cc:824] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:44.491919 7597 ts_tablet_manager.cc:1428] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:21:44.495726 7600 tablet_bootstrap.cc:492] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: No bootstrap required, opened a new log
I20260811 22:21:44.495962 7600 ts_tablet_manager.cc:1397] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:21:44.499931 7600 raft_consensus.cc:348] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.500242 7600 raft_consensus.cc:374] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:44.500324 7600 raft_consensus.cc:729] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84c5ba6d5184657bd4a9c17a5765a80, State: Initialized, Role: FOLLOWER
I20260811 22:21:44.500828 7600 consensus_queue.cc:260] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.501649 7545 heartbeater.cc:503] Master 127.26.125.62:45967 was elected leader, sending a full tablet report...
I20260811 22:21:44.502046 7600 ts_tablet_manager.cc:1428] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 22:21:44.652961 7550 tablet.cc:2369] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:21:44.687285 7323 tablet.cc:2369] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:21:44.703742 7081 tablet.cc:2369] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:44.789368 7602 raft_consensus.cc:482] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:44.789647 7602 raft_consensus.cc:504] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.790971 7602 leader_election.cc:288] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713), c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661)
I20260811 22:21:44.795482 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.791225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:34713 (local address 127.26.125.2:55953)
0811 22:21:44.791618 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.791634 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.791772 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.792051 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.792054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.792080 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:44.792257 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.792261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.793335 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.793338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.794216 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.794228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.795050 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.795054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.795146 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.795176 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.795256 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.795336 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":17,"spinlock_wait_cycles":17792,"thread_start_us":77,"threads_started":1}
I20260811 22:21:44.795483 7587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.791315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:55953 (local address 127.26.125.1:34713)
0811 22:21:44.791454 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.791459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.791722 (+ 263us) server_negotiation.cc:407] Connection header received
0811 22:21:44.791871 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.791874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.791932 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.792018 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:44.792362 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.793205 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.794373 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.794842 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.795026 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.795086 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.795273 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.795353 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:21:44.796357 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.791225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:45661 (local address 127.26.125.2:37627)
0811 22:21:44.791626 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:44.791637 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:44.791772 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:44.792056 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:44.792059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:44.792080 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:44.792253 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.792261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.794347 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.794350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.795210 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:44.795217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.796055 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.796058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:44.796127 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.796147 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:21:44.796205 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 22:21:44.796254 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":63,"spinlock_wait_cycles":8192,"thread_start_us":66,"threads_started":1}
I20260811 22:21:44.796268 6992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d3afcaa18941f4a4843fedf1dee3fb" candidate_uuid: "13fbdc730e1446b8937ff734023d6305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde0b8d5e9a74062ba71967366d77a28" is_pre_election: true
I20260811 22:21:44.796419 7589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.791427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:37627 (local address 127.26.125.3:45661)
0811 22:21:44.791542 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.791545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.791720 (+ 175us) server_negotiation.cc:407] Connection header received
0811 22:21:44.791871 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.791874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.791929 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.792018 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:44.792362 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.793205 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.795325 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.795932 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.796059 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.796108 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.796232 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.796310 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:21:44.796644 6992 raft_consensus.cc:2454] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fbdc730e1446b8937ff734023d6305 in term 0.
I20260811 22:21:44.797053 7467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d3afcaa18941f4a4843fedf1dee3fb" candidate_uuid: "13fbdc730e1446b8937ff734023d6305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" is_pre_election: true
I20260811 22:21:44.797120 7125 leader_election.cc:302] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fbdc730e1446b8937ff734023d6305, bde0b8d5e9a74062ba71967366d77a28; no voters:
I20260811 22:21:44.797341 7467 raft_consensus.cc:2454] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fbdc730e1446b8937ff734023d6305 in term 0.
I20260811 22:21:44.797389 7602 raft_consensus.cc:2781] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:44.797492 7602 raft_consensus.cc:482] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:44.797550 7602 raft_consensus.cc:3037] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:44.798650 7602 raft_consensus.cc:504] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.799059 7602 leader_election.cc:288] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [CANDIDATE]: Term 1 election: Requested vote from peers bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713), c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661)
I20260811 22:21:44.799402 6992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d3afcaa18941f4a4843fedf1dee3fb" candidate_uuid: "13fbdc730e1446b8937ff734023d6305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde0b8d5e9a74062ba71967366d77a28"
I20260811 22:21:44.799474 7467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d3afcaa18941f4a4843fedf1dee3fb" candidate_uuid: "13fbdc730e1446b8937ff734023d6305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84c5ba6d5184657bd4a9c17a5765a80"
I20260811 22:21:44.799556 6992 raft_consensus.cc:3037] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:44.799587 7467 raft_consensus.cc:3037] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:44.800920 6992 raft_consensus.cc:2454] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fbdc730e1446b8937ff734023d6305 in term 1.
I20260811 22:21:44.800920 7467 raft_consensus.cc:2454] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fbdc730e1446b8937ff734023d6305 in term 1.
I20260811 22:21:44.801215 7125 leader_election.cc:302] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fbdc730e1446b8937ff734023d6305, bde0b8d5e9a74062ba71967366d77a28; no voters:
I20260811 22:21:44.801395 7602 raft_consensus.cc:2781] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:44.801585 7602 raft_consensus.cc:686] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 1 LEADER]: Becoming Leader. State: Replica: 13fbdc730e1446b8937ff734023d6305, State: Running, Role: LEADER
I20260811 22:21:44.801956 7602 consensus_queue.cc:237] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:44.805709 5373 catalog_manager.cc:5495] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13fbdc730e1446b8937ff734023d6305 (127.26.125.2). New cstate: current_term: 1 leader_uuid: "13fbdc730e1446b8937ff734023d6305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } health_report { overall_health: HEALTHY } } }
I20260811 22:21:44.842443 7591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.839119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.26.125.2:34859)
0811 22:21:44.839244 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.839248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.839260 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:21:44.839344 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.839346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.839401 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.839493 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:44.839776 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.840268 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.841061 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.841270 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.841338 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.841673 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:44.842124 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:44.842220 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.842269 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.842324 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:21:44.848887 7467 raft_consensus.cc:1261] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 1 FOLLOWER]: Refusing update from remote peer 13fbdc730e1446b8937ff734023d6305: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:44.848876 6992 raft_consensus.cc:1261] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 1 FOLLOWER]: Refusing update from remote peer 13fbdc730e1446b8937ff734023d6305: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:44.849833 7612 consensus_queue.cc:1035] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:44.850070 7602 consensus_queue.cc:1035] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:44.859932 7617 mvcc.cc:205] Tried to move back new op lower bound from 7317450362252427264 to 7317450362071420928. Current Snapshot: MvccSnapshot[applied={T|T < 7317450362252427264}]
I20260811 22:21:44.871915 5374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:44.872094 5374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:44.876876 5374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bee3d139283d4c78966aa9c23c82c6ca: Sending DeleteTablet for 3 replicas of tablet 13d3afcaa18941f4a4843fedf1dee3fb
I20260811 22:21:44.877959 6956 tablet_service.cc:1514] Processing DeleteTablet for tablet 13d3afcaa18941f4a4843fedf1dee3fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:44 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:34438
I20260811 22:21:44.877995 7436 tablet_service.cc:1514] Processing DeleteTablet for tablet 13d3afcaa18941f4a4843fedf1dee3fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:44 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:39294
I20260811 22:21:44.878399 7208 tablet_service.cc:1514] Processing DeleteTablet for tablet 13d3afcaa18941f4a4843fedf1dee3fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:44 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:60590
I20260811 22:21:44.878612 7638 tablet_replica.cc:330] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: stopping tablet replica
I20260811 22:21:44.878921 7640 tablet_replica.cc:330] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: stopping tablet replica
I20260811 22:21:44.879016 7638 raft_consensus.cc:2229] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:44.879339 7640 raft_consensus.cc:2229] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:44.879392 7638 raft_consensus.cc:2258] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:44.879868 7640 raft_consensus.cc:2258] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:44.881335 7638 ts_tablet_manager.cc:1905] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:44.881497 7640 ts_tablet_manager.cc:1905] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:44.882831 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 5309
I20260811 22:21:44.897853 7640 ts_tablet_manager.cc:1918] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:44.898138 7640 log.cc:1196] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/13d3afcaa18941f4a4843fedf1dee3fb
I20260811 22:21:44.898685 7640 ts_tablet_manager.cc:1939] T 13d3afcaa18941f4a4843fedf1dee3fb P 13fbdc730e1446b8937ff734023d6305: Deleting consensus metadata
I20260811 22:21:44.902966 7638 ts_tablet_manager.cc:1918] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:44.903136 7638 log.cc:1196] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/13d3afcaa18941f4a4843fedf1dee3fb
I20260811 22:21:44.903625 7638 ts_tablet_manager.cc:1939] T 13d3afcaa18941f4a4843fedf1dee3fb P c84c5ba6d5184657bd4a9c17a5765a80: Deleting consensus metadata
I20260811 22:21:44.907588 7637 tablet_replica.cc:330] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: stopping tablet replica
I20260811 22:21:44.908072 7637 raft_consensus.cc:2229] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:44.908504 7637 raft_consensus.cc:2258] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:44.910652 7637 ts_tablet_manager.cc:1905] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:21:44.913775 7130 connection.cc:535] server connection from 127.0.0.1:60590 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:21:44.913964 7637 ts_tablet_manager.cc:1918] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:44.914094 7637 log.cc:1196] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/13d3afcaa18941f4a4843fedf1dee3fb
W20260811 22:21:44.914445 7357 connection.cc:535] server connection from 127.0.0.1:39294 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:21:44.914480 7637 ts_tablet_manager.cc:1939] T 13d3afcaa18941f4a4843fedf1dee3fb P bde0b8d5e9a74062ba71967366d77a28: Deleting consensus metadata
W20260811 22:21:44.915946 6878 connection.cc:441] server connection from 127.0.0.1:34438 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34438 (request call id 1) could send its response
I20260811 22:21:44.917069 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.880987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.26.125.61:36207)
0811 22:21:44.881265 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.881270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.907251 (+ 25981us) server_negotiation.cc:407] Connection header received
0811 22:21:44.907386 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.907389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.907450 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.907541 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.908671 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.909435 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.910682 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.911061 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.911150 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.913158 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.913162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.913165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.913225 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.915094 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.915652 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.915656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.915658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.915759 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.916081 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.916085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.916086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.916322 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.916456 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.916779 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.916910 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260811 22:21:44.926688 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.881128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.26.125.60:41171)
0811 22:21:44.881373 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.881378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.917133 (+ 35755us) server_negotiation.cc:407] Connection header received
0811 22:21:44.917317 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.917321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.917378 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.917444 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.918145 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.918700 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.919716 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.920072 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.920191 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.922264 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.922269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.922271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.922303 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.924007 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.924501 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.924505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.924519 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.924625 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.924915 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.924918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.924920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.925137 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.925253 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.925938 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.926495 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260811 22:21:44.931228 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.920399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.26.125.60:41171)
0811 22:21:44.920971 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.920975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.921660 (+ 685us) server_negotiation.cc:407] Connection header received
0811 22:21:44.921706 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.921708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.921763 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.921842 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.923296 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.924080 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.925228 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.925543 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.925630 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.927709 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.927713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.927716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.927740 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.929415 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.929902 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.929905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.929907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.929988 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.930280 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.930283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.930285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.930504 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.930631 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.930846 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.930999 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":75,"threads_started":1}
I20260811 22:21:44.931416 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.920554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.26.125.61:36207)
0811 22:21:44.920691 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.920696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.921285 (+ 589us) server_negotiation.cc:407] Connection header received
0811 22:21:44.921650 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.921653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.921701 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.921749 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.923114 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.923922 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.925139 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.925478 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.925560 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.927631 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.927635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.927638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.927666 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.929290 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.929899 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.929903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.929904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.929994 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.930332 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.930336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.930338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.930613 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.930715 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.931121 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.931257 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:45.863307 7660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:45.862638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.2:36767)
0811 22:21:45.863060 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:45.863181 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":96,"threads_started":1}
W20260811 22:21:45.863659 7320 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:45967 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
I20260811 22:21:45.864295 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:45.863900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.1:49911)
0811 22:21:45.864131 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:45.864211 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
W20260811 22:21:45.864651 7077 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:45967 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
I20260811 22:21:45.866353 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:45.865926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.3:53499)
0811 22:21:45.866171 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:45.866255 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
W20260811 22:21:45.866663 7545 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:45967 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
I20260811 22:21:46.376720 7667 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bee3d139283d4c78966aa9c23c82c6ca)
I20260811 22:21:46.376983 7667 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } }
I20260811 22:21:46.377324 7668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bee3d139283d4c78966aa9c23c82c6ca)
I20260811 22:21:46.377554 7668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } }
I20260811 22:21:46.378170 7667 leader_election.cc:288] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bee3d139283d4c78966aa9c23c82c6ca (127.26.125.62:45967), 2587ac2153bd4418bbb8b86a8184cde2 (127.26.125.61:36207)
I20260811 22:21:46.379038 7668 leader_election.cc:288] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bee3d139283d4c78966aa9c23c82c6ca (127.26.125.62:45967), e8150f1dce374c6c979f79e3220374ff (127.26.125.60:41171)
I20260811 22:21:46.379180 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.378439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48770)
0811 22:21:46.378973 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.379068 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":81,"threads_started":1}
W20260811 22:21:46.379365 6737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
I20260811 22:21:46.379824 7677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.378872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48780)
0811 22:21:46.379648 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.379728 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":62,"threads_started":1}
W20260811 22:21:46.380007 6408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
I20260811 22:21:46.380465 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.380176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48790)
0811 22:21:46.380326 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.380380 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 22:21:46.380631 6737 leader_election.cc:334] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bee3d139283d4c78966aa9c23c82c6ca (127.26.125.62:45967): Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
I20260811 22:21:46.381045 7677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.380839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48800)
0811 22:21:46.380930 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.380981 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 22:21:46.381179 6408 leader_election.cc:334] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bee3d139283d4c78966aa9c23c82c6ca (127.26.125.62:45967): Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
I20260811 22:21:46.382836 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.378430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:36207 (local address 127.0.0.1:54098)
0811 22:21:46.378769 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.378785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:46.378945 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:46.379351 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:46.379355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:46.379378 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:46.379487 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.379495 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.380502 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.380517 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.381615 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.381625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.382292 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.382296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.382392 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.382524 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 22:21:46.382596 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:21:46.382692 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":78,"thread_start_us":83,"threads_started":1}
I20260811 22:21:46.382875 7673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.378539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.26.125.61:36207)
0811 22:21:46.378804 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:46.378809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:46.379037 (+ 228us) server_negotiation.cc:407] Connection header received
0811 22:21:46.379090 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:46.379093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:46.379142 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:46.379217 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:46.379629 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.380384 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.381737 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.382171 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.382267 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.382335 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:46.382634 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 22:21:46.382717 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260811 22:21:46.383407 6465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8150f1dce374c6c979f79e3220374ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2587ac2153bd4418bbb8b86a8184cde2" is_pre_election: true
I20260811 22:21:46.383654 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.379270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.26.125.60:41171)
0811 22:21:46.379568 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:46.379574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:46.379585 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:21:46.379622 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:46.379624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:46.379663 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:46.379734 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:46.380247 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.381238 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.382311 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.382800 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.382939 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.383012 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:46.383455 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 22:21:46.383542 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":237,"thread_start_us":74,"threads_started":1}
I20260811 22:21:46.383668 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.378812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41171 (local address 127.0.0.1:42264)
0811 22:21:46.379241 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.379255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:46.379353 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:46.379831 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:46.379835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:46.379853 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:46.380076 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.380082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.381353 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.381356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.382175 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.382185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.383220 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.383224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.383322 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.383344 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:21:46.383417 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:21:46.383514 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":105,"threads_started":1}
I20260811 22:21:46.383697 6465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8150f1dce374c6c979f79e3220374ff in term 1.
I20260811 22:21:46.384124 6738 leader_election.cc:302] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2587ac2153bd4418bbb8b86a8184cde2, e8150f1dce374c6c979f79e3220374ff; no voters: bee3d139283d4c78966aa9c23c82c6ca
I20260811 22:21:46.384202 6781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2587ac2153bd4418bbb8b86a8184cde2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e8150f1dce374c6c979f79e3220374ff" is_pre_election: true
I20260811 22:21:46.384420 7667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:21:46.384492 7667 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Starting leader election (detected failure of leader bee3d139283d4c78966aa9c23c82c6ca)
I20260811 22:21:46.384532 7667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:21:46.385706 7667 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } }
I20260811 22:21:46.385985 6781 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2587ac2153bd4418bbb8b86a8184cde2 in current term 2: Already voted for candidate e8150f1dce374c6c979f79e3220374ff in this term.
I20260811 22:21:46.386170 7667 leader_election.cc:288] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [CANDIDATE]: Term 2 election: Requested vote from peers bee3d139283d4c78966aa9c23c82c6ca (127.26.125.62:45967), 2587ac2153bd4418bbb8b86a8184cde2 (127.26.125.61:36207)
I20260811 22:21:46.386610 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.386322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48808)
0811 22:21:46.386447 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.386529 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 22:21:46.386631 6465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8150f1dce374c6c979f79e3220374ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2587ac2153bd4418bbb8b86a8184cde2"
I20260811 22:21:46.386783 6465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:21:46.386783 6413 leader_election.cc:302] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2587ac2153bd4418bbb8b86a8184cde2; no voters: bee3d139283d4c78966aa9c23c82c6ca, e8150f1dce374c6c979f79e3220374ff
I20260811 22:21:46.387406 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.387110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48818)
0811 22:21:46.387306 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.387349 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260811 22:21:46.387526 6737 leader_election.cc:334] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bee3d139283d4c78966aa9c23c82c6ca (127.26.125.62:45967): Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
I20260811 22:21:46.388064 6465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8150f1dce374c6c979f79e3220374ff in term 2.
I20260811 22:21:46.388185 7668 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:21:46.388437 6738 leader_election.cc:302] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2587ac2153bd4418bbb8b86a8184cde2, e8150f1dce374c6c979f79e3220374ff; no voters: bee3d139283d4c78966aa9c23c82c6ca
I20260811 22:21:46.388751 7667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:21:46.389010 7667 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [term 2 LEADER]: Becoming Leader. State: Replica: e8150f1dce374c6c979f79e3220374ff, State: Running, Role: LEADER
I20260811 22:21:46.389292 7667 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } }
I20260811 22:21:46.390583 7688 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8150f1dce374c6c979f79e3220374ff. Latest consensus state: current_term: 2 leader_uuid: "e8150f1dce374c6c979f79e3220374ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:46.390715 7688 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:46.391063 7690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:46.393639 7690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:46.394084 7690 catalog_manager.cc:1259] Loaded cluster ID: 357565593f734e1ca5de1813dad5b7a2
I20260811 22:21:46.394134 7690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:46.394726 7690 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:46.395136 7690 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Loaded TSK: 0
I20260811 22:21:46.395879 7690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:21:46.453387 6762 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:46.456988 6762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:46.460480 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.460165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48858)
0811 22:21:46.460295 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.460368 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 22:21:46.460525 6465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [term 2 FOLLOWER]: Refusing update from remote peer e8150f1dce374c6c979f79e3220374ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 22:21:46.461114 7667 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 22:21:46.461206 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.460993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.0.0.1:48874)
0811 22:21:46.461097 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.461146 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 22:21:46.461453 6737 consensus_peers.cc:487] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff -> Peer bee3d139283d4c78966aa9c23c82c6ca (127.26.125.62:45967): Couldn't send request to peer bee3d139283d4c78966aa9c23c82c6ca. Status: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:21:46.462699 7668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8150f1dce374c6c979f79e3220374ff. Latest consensus state: current_term: 2 leader_uuid: "e8150f1dce374c6c979f79e3220374ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:46.462841 7668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:46.464062 7688 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "e8150f1dce374c6c979f79e3220374ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:46.464202 7688 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:46.464530 7668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "e8150f1dce374c6c979f79e3220374ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bee3d139283d4c78966aa9c23c82c6ca" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45967 } } peers { permanent_uuid: "2587ac2153bd4418bbb8b86a8184cde2" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 36207 } attrs { promote: false } } peers { permanent_uuid: "e8150f1dce374c6c979f79e3220374ff" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41171 } attrs { promote: false } } }
I20260811 22:21:46.464663 7668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2587ac2153bd4418bbb8b86a8184cde2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:46.475245 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.471117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.26.125.1:34713)
0811 22:21:46.471826 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:46.471830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:46.471844 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:46.471894 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:46.471897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:46.471944 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:46.472307 (+ 363us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:46.472419 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.473042 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.474123 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.474676 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.474783 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.474888 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:46.475037 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 22:21:46.475116 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":79,"threads_started":1}
I20260811 22:21:46.475244 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.470614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:34713 (local address 127.0.0.1:34450)
0811 22:21:46.470701 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.470712 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:46.470766 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:46.472084 (+ 1318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:46.472088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:46.472107 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:46.472201 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.472207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.473142 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.473146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.473976 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.473984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.474798 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.474802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.474918 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.474945 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:21:46.475020 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:21:46.475099 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260811 22:21:46.475605 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.470492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:45661 (local address 127.0.0.1:39310)
0811 22:21:46.470604 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.470619 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:46.470712 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:46.471172 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:46.471174 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:46.471193 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:46.471314 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.471321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.472809 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.472812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.474012 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.474023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.474946 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.474949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.475039 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.475066 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:21:46.475115 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 22:21:46.475183 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 22:21:46.475630 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.470694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39310 (local address 127.26.125.3:45661)
0811 22:21:46.470958 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:46.470962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:46.470977 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:46.471032 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:46.471034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:46.471083 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:46.471167 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:46.471448 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.472705 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.474154 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.474779 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.475321 (+ 542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.475392 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:46.475431 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 22:21:46.475501 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":99,"threads_started":1}
I20260811 22:21:46.476042 6956 tablet_service.cc:1467] Processing CreateTablet for tablet 76f65cf7eb5b4ed2822c00d36049e568 (DEFAULT_TABLE table=test-table [id=d53943515ad84ef99c5c2db304eaee90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:46.476329 6956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f65cf7eb5b4ed2822c00d36049e568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:46.478438 7706 tablet_bootstrap.cc:492] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28: Bootstrap starting.
I20260811 22:21:46.478941 7436 tablet_service.cc:1467] Processing CreateTablet for tablet 76f65cf7eb5b4ed2822c00d36049e568 (DEFAULT_TABLE table=test-table [id=d53943515ad84ef99c5c2db304eaee90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:46.479228 7436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f65cf7eb5b4ed2822c00d36049e568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:46.479529 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.471697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:34859 (local address 127.0.0.1:60610)
0811 22:21:46.471941 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.471953 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:46.472043 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:46.472519 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:46.472523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:46.472539 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:46.472642 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.472647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.473646 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.473650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.474772 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.474781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.479106 (+ 4325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.479109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:46.479209 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.479232 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:21:46.479301 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:21:46.479386 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
I20260811 22:21:46.479532 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.472081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.26.125.2:34859)
0811 22:21:46.472311 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:46.472315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:46.472326 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:21:46.472379 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:46.472381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:46.472417 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:46.472491 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:46.472743 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.473515 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.474948 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.478978 (+ 4030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.479095 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.479162 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:46.479326 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 22:21:46.479418 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260811 22:21:46.480120 7208 tablet_service.cc:1467] Processing CreateTablet for tablet 76f65cf7eb5b4ed2822c00d36049e568 (DEFAULT_TABLE table=test-table [id=d53943515ad84ef99c5c2db304eaee90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:46.480367 7208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f65cf7eb5b4ed2822c00d36049e568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:46.481535 7707 tablet_bootstrap.cc:492] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: Bootstrap starting.
I20260811 22:21:46.482218 7706 tablet_bootstrap.cc:654] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:46.482429 7707 tablet_bootstrap.cc:654] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:46.482802 7709 tablet_bootstrap.cc:492] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: Bootstrap starting.
I20260811 22:21:46.483629 7706 tablet_bootstrap.cc:492] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28: No bootstrap required, opened a new log
I20260811 22:21:46.483740 7706 ts_tablet_manager.cc:1397] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:21:46.483748 7709 tablet_bootstrap.cc:654] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:46.483940 7707 tablet_bootstrap.cc:492] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: No bootstrap required, opened a new log
I20260811 22:21:46.484033 7707 ts_tablet_manager.cc:1397] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 22:21:46.484263 7706 raft_consensus.cc:348] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.484388 7706 raft_consensus.cc:374] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:46.484421 7706 raft_consensus.cc:729] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde0b8d5e9a74062ba71967366d77a28, State: Initialized, Role: FOLLOWER
I20260811 22:21:46.484498 7707 raft_consensus.cc:348] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.484622 7707 raft_consensus.cc:374] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:46.484591 7706 consensus_queue.cc:260] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.484652 7707 raft_consensus.cc:729] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84c5ba6d5184657bd4a9c17a5765a80, State: Initialized, Role: FOLLOWER
I20260811 22:21:46.484760 7707 consensus_queue.cc:260] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.484917 7706 ts_tablet_manager.cc:1428] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:21:46.485103 7707 ts_tablet_manager.cc:1428] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:21:46.485244 7709 tablet_bootstrap.cc:492] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: No bootstrap required, opened a new log
I20260811 22:21:46.485327 7709 ts_tablet_manager.cc:1397] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:46.486873 7709 raft_consensus.cc:348] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.487005 7709 raft_consensus.cc:374] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:46.487036 7709 raft_consensus.cc:729] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fbdc730e1446b8937ff734023d6305, State: Initialized, Role: FOLLOWER
I20260811 22:21:46.487114 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.485537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.3:37563)
0811 22:21:46.486932 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.486999 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1298,"thread_start_us":105,"threads_started":1}
I20260811 22:21:46.487188 7709 consensus_queue.cc:260] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.487545 7709 ts_tablet_manager.cc:1428] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:46.487884 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.487529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.3:55671)
0811 22:21:46.487774 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.487815 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":85,"threads_started":1}
I20260811 22:21:46.488242 7543 heartbeater.cc:503] Master 127.26.125.60:41171 was elected leader, sending a full tablet report...
I20260811 22:21:46.489537 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.486197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.1:49863)
0811 22:21:46.489365 (+ 3168us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.489428 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3103,"thread_start_us":97,"threads_started":1}
I20260811 22:21:46.489956 7076 heartbeater.cc:503] Master 127.26.125.60:41171 was elected leader, sending a full tablet report...
I20260811 22:21:46.490291 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.488757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.2:34585)
0811 22:21:46.490140 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.490197 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1305,"thread_start_us":108,"threads_started":1}
I20260811 22:21:46.490406 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.490164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.1:51059)
0811 22:21:46.490303 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.490349 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 22:21:46.490650 7319 heartbeater.cc:503] Master 127.26.125.60:41171 was elected leader, sending a full tablet report...
I20260811 22:21:46.491369 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.491124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.2:36559)
0811 22:21:46.491257 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.491306 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 22:21:46.591282 7720 raft_consensus.cc:482] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:46.591502 7720 raft_consensus.cc:504] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.591993 7720 leader_election.cc:288] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661), bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713)
I20260811 22:21:46.592403 7467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f65cf7eb5b4ed2822c00d36049e568" candidate_uuid: "13fbdc730e1446b8937ff734023d6305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" is_pre_election: true
I20260811 22:21:46.592486 6992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f65cf7eb5b4ed2822c00d36049e568" candidate_uuid: "13fbdc730e1446b8937ff734023d6305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde0b8d5e9a74062ba71967366d77a28" is_pre_election: true
I20260811 22:21:46.592590 7467 raft_consensus.cc:2454] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fbdc730e1446b8937ff734023d6305 in term 0.
I20260811 22:21:46.592630 6992 raft_consensus.cc:2454] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fbdc730e1446b8937ff734023d6305 in term 0.
I20260811 22:21:46.592962 7130 leader_election.cc:302] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fbdc730e1446b8937ff734023d6305, c84c5ba6d5184657bd4a9c17a5765a80; no voters:
I20260811 22:21:46.593273 7720 raft_consensus.cc:2781] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:46.593340 7720 raft_consensus.cc:482] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:46.593454 7720 raft_consensus.cc:3037] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:46.594620 7720 raft_consensus.cc:504] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.594998 7720 leader_election.cc:288] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [CANDIDATE]: Term 1 election: Requested vote from peers c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661), bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713)
I20260811 22:21:46.595383 7467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f65cf7eb5b4ed2822c00d36049e568" candidate_uuid: "13fbdc730e1446b8937ff734023d6305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84c5ba6d5184657bd4a9c17a5765a80"
I20260811 22:21:46.595407 6992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f65cf7eb5b4ed2822c00d36049e568" candidate_uuid: "13fbdc730e1446b8937ff734023d6305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde0b8d5e9a74062ba71967366d77a28"
I20260811 22:21:46.595504 7467 raft_consensus.cc:3037] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:46.595531 6992 raft_consensus.cc:3037] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:46.596431 6992 raft_consensus.cc:2454] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fbdc730e1446b8937ff734023d6305 in term 1.
I20260811 22:21:46.596437 7467 raft_consensus.cc:2454] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fbdc730e1446b8937ff734023d6305 in term 1.
I20260811 22:21:46.596724 7130 leader_election.cc:302] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13fbdc730e1446b8937ff734023d6305, bde0b8d5e9a74062ba71967366d77a28, c84c5ba6d5184657bd4a9c17a5765a80; no voters:
I20260811 22:21:46.596904 7720 raft_consensus.cc:2781] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:46.597134 7720 raft_consensus.cc:686] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 1 LEADER]: Becoming Leader. State: Replica: 13fbdc730e1446b8937ff734023d6305, State: Running, Role: LEADER
I20260811 22:21:46.597496 7720 consensus_queue.cc:237] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } }
I20260811 22:21:46.597935 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.597657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.2:46535)
0811 22:21:46.597772 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.597837 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 22:21:46.599659 6762 catalog_manager.cc:5495] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13fbdc730e1446b8937ff734023d6305 (127.26.125.2). New cstate: current_term: 1 leader_uuid: "13fbdc730e1446b8937ff734023d6305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fbdc730e1446b8937ff734023d6305" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34859 } health_report { overall_health: HEALTHY } } }
I20260811 22:21:46.629591 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.626546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.26.125.2:34859)
0811 22:21:46.626689 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:46.626694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:46.626711 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:21:46.626764 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:46.626766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:46.626809 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:46.626894 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:46.627151 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.627663 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.628528 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.628746 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.628833 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:46.629126 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:46.629204 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:46.629262 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:46.629363 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 22:21:46.629443 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:21:46.631973 6992 raft_consensus.cc:1261] T 76f65cf7eb5b4ed2822c00d36049e568 P bde0b8d5e9a74062ba71967366d77a28 [term 1 FOLLOWER]: Refusing update from remote peer 13fbdc730e1446b8937ff734023d6305: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:46.631992 7467 raft_consensus.cc:1261] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 1 FOLLOWER]: Refusing update from remote peer 13fbdc730e1446b8937ff734023d6305: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:46.632452 7720 consensus_queue.cc:1035] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bde0b8d5e9a74062ba71967366d77a28" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:46.632645 7724 consensus_queue.cc:1035] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84c5ba6d5184657bd4a9c17a5765a80" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:46.634092 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.633823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.1:48643)
0811 22:21:46.633943 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.634009 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 22:21:46.636260 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.635989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.3:47173)
0811 22:21:46.636093 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.636151 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 22:21:46.638320 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.638061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.2:46037)
0811 22:21:46.638179 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.638239 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 22:21:46.639092 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.638787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.1:51895)
0811 22:21:46.638956 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.639011 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 22:21:46.639758 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.639506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.3:59703)
0811 22:21:46.639643 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.639692 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 22:21:46.641387 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.640987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45967 (local address 127.26.125.2:40187)
0811 22:21:46.641261 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.641317 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260811 22:21:46.645630 6762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:46.645785 6762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:46.649559 6762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8150f1dce374c6c979f79e3220374ff: Sending DeleteTablet for 3 replicas of tablet 76f65cf7eb5b4ed2822c00d36049e568
I20260811 22:21:46.650454 7208 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f65cf7eb5b4ed2822c00d36049e568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:46 UTC) from {username='oryx', principal='oryx/127.26.125.60@KRBTEST.COM'} at 127.0.0.1:60610
I20260811 22:21:46.650529 6956 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f65cf7eb5b4ed2822c00d36049e568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:46 UTC) from {username='oryx', principal='oryx/127.26.125.60@KRBTEST.COM'} at 127.0.0.1:34450
I20260811 22:21:46.650895 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 6696
I20260811 22:21:46.653687 7436 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f65cf7eb5b4ed2822c00d36049e568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:46 UTC) from {username='oryx', principal='oryx/127.26.125.60@KRBTEST.COM'} at 127.0.0.1:39310
I20260811 22:21:46.667375 7751 tablet_replica.cc:330] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: stopping tablet replica
I20260811 22:21:46.667349 7749 tablet_replica.cc:330] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: stopping tablet replica
I20260811 22:21:46.667539 7751 raft_consensus.cc:2229] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:46.667543 7749 raft_consensus.cc:2229] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:46.667747 7751 raft_consensus.cc:2258] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:46.667806 7749 raft_consensus.cc:2258] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:46.668658 7749 ts_tablet_manager.cc:1905] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:46.671234 7749 ts_tablet_manager.cc:1918] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:46.671314 7749 log.cc:1196] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/76f65cf7eb5b4ed2822c00d36049e568
I20260811 22:21:46.671576 7749 ts_tablet_manager.cc:1939] T 76f65cf7eb5b4ed2822c00d36049e568 P 13fbdc730e1446b8937ff734023d6305: Deleting consensus metadata
I20260811 22:21:46.672385 6736 catalog_manager.cc:4841] TS 13fbdc730e1446b8937ff734023d6305 (127.26.125.2:34859): tablet 76f65cf7eb5b4ed2822c00d36049e568 (table test-table [id=d53943515ad84ef99c5c2db304eaee90]) successfully deleted
I20260811 22:21:46.673123 7751 ts_tablet_manager.cc:1905] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:46.673765 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 7091
I20260811 22:21:46.674191 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.673855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:34713 (local address 127.0.0.1:53434)
0811 22:21:46.674041 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.674098 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:34713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260811 22:21:46.674451 6737 catalog_manager.cc:4584] TS bde0b8d5e9a74062ba71967366d77a28 (127.26.125.1:34713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 76f65cf7eb5b4ed2822c00d36049e568: Network error: Client connection negotiation failed: client connection to 127.26.125.1:34713: connect: Connection refused (error 111)
I20260811 22:21:46.676724 7751 ts_tablet_manager.cc:1918] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:46.676800 7751 log.cc:1196] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/76f65cf7eb5b4ed2822c00d36049e568
I20260811 22:21:46.677026 7751 ts_tablet_manager.cc:1939] T 76f65cf7eb5b4ed2822c00d36049e568 P c84c5ba6d5184657bd4a9c17a5765a80: Deleting consensus metadata
I20260811 22:21:46.680913 6739 catalog_manager.cc:4841] TS c84c5ba6d5184657bd4a9c17a5765a80 (127.26.125.3:45661): tablet 76f65cf7eb5b4ed2822c00d36049e568 (table test-table [id=d53943515ad84ef99c5c2db304eaee90]) successfully deleted
I20260811 22:21:46.688544 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 7328
I20260811 22:21:46.710732 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 6176
I20260811 22:21:46.723282 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 6366
2026-08-11T22:21:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6001 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:46 dist-test-slave-5bn2 krb5kdc[7760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:46 dist-test-slave-5bn2 krb5kdc[7760](info): set up 2 sockets
Aug 11 22:21:46 dist-test-slave-5bn2 krb5kdc[7760](info): commencing operation
krb5kdc: starting...
W20260811 22:21:47.917357 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.62.keytab.
Entry for principal oryx/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.62.keytab.
I20260811 22:21:48.058254 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41983
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.62.keytab
--principal=oryx/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:41983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:48.177958 7787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:48.178293 7787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:48.178355 7787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:48.182505 7787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:48.182592 7787 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:21:48.182616 7787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:48.182639 7787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:48.182659 7787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:48.182682 7787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:48.188278 7787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:41983
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.62.keytab
--principal=oryx/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:48.189478 7787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:48.190588 7787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:48.199249 7795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.199249 7798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.199229 7796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:48.199615 7787 server_base.cc:1034] running on GCE node
I20260811 22:21:48.200255 7787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:48.201483 7787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:48.202670 7787 hybrid_clock.cc:648] HybridClock initialized: now 1786486908202636 us; error 49 us; skew 500 ppm
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:48.205696 7787 init.cc:377] Logged in from keytab as oryx/127.26.125.62@KRBTEST.COM (short username oryx)
I20260811 22:21:48.206708 7787 webserver.cc:460] Webserver started at http://127.26.125.62:40223/ using document root <none> and password file <none>
I20260811 22:21:48.207343 7787 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:48.207417 7787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:48.207633 7787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:48.209805 7787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "43cac8993a264ab8a76b1e83267f3c5c"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "dc4cb0b54118cdf4b6cdf86636755cb6"
server_key_iv: "00a33480306e434da330e34772dc5d9e"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.210403 7787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "43cac8993a264ab8a76b1e83267f3c5c"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "dc4cb0b54118cdf4b6cdf86636755cb6"
server_key_iv: "00a33480306e434da330e34772dc5d9e"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.214406 7787 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:21:48.217207 7813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.218523 7787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:21:48.218647 7787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "43cac8993a264ab8a76b1e83267f3c5c"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "dc4cb0b54118cdf4b6cdf86636755cb6"
server_key_iv: "00a33480306e434da330e34772dc5d9e"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.218755 7787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:48.233081 7787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:48.236641 7787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:48.236860 7787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:48.245363 7787 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:41983
I20260811 22:21:48.245373 7912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:41983 every 8 connection(s)
I20260811 22:21:48.246428 7787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:48.246680 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 7787
I20260811 22:21:48.246814 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:48.247191 27124 external_mini_cluster.cc:1428] Setting key f6669a9f6b32e7de9ce7d24c1c5f769c
I20260811 22:21:48.249891 7914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:48.255959 7914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: Bootstrap starting.
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:48.258834 7914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:48.260213 7914 log.cc:824] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:48.262660 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.248786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57808 (local address 127.26.125.62:41983)
0811 22:21:48.249302 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.249319 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.249415 (+ 96us) server_negotiation.cc:407] Connection header received
0811 22:21:48.250136 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.250158 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.250542 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.250926 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.251811 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.252852 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.253710 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.254198 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.254345 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.256878 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.256884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.256902 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.256938 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.259058 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.259650 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.259657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.259663 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.259816 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.260143 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.260146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.260148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.260737 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.260893 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.261289 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.261674 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":164,"threads_started":1}
I20260811 22:21:48.262797 7914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: No bootstrap required, opened a new log
I20260811 22:21:48.266614 7914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cac8993a264ab8a76b1e83267f3c5c" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41983 } }
I20260811 22:21:48.266947 7914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:48.267005 7914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cac8993a264ab8a76b1e83267f3c5c, State: Initialized, Role: FOLLOWER
I20260811 22:21:48.267446 7914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cac8993a264ab8a76b1e83267f3c5c" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41983 } }
I20260811 22:21:48.267581 7914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:48.267629 7914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:48.267707 7914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:48.268707 7914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cac8993a264ab8a76b1e83267f3c5c" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41983 } }
I20260811 22:21:48.269037 7914 leader_election.cc:302] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43cac8993a264ab8a76b1e83267f3c5c; no voters:
I20260811 22:21:48.269345 7914 leader_election.cc:288] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:48.269503 7921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:48.269753 7921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [term 1 LEADER]: Becoming Leader. State: Replica: 43cac8993a264ab8a76b1e83267f3c5c, State: Running, Role: LEADER
I20260811 22:21:48.270152 7921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cac8993a264ab8a76b1e83267f3c5c" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41983 } }
I20260811 22:21:48.270709 7914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:48.272105 7922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43cac8993a264ab8a76b1e83267f3c5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cac8993a264ab8a76b1e83267f3c5c" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41983 } } }
I20260811 22:21:48.272225 7922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:48.272569 7926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43cac8993a264ab8a76b1e83267f3c5c. Latest consensus state: current_term: 1 leader_uuid: "43cac8993a264ab8a76b1e83267f3c5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cac8993a264ab8a76b1e83267f3c5c" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41983 } } }
I20260811 22:21:48.272686 7926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:48.272722 7938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:48.275723 7938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:48.282953 7938 catalog_manager.cc:1347] Generated new cluster ID: 3155cc114ac04ece9bb81c2a34c7ec0a
I20260811 22:21:48.283074 7938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:48.299220 7938 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:48.300300 7938 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:48.307906 7938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: Generated new TSK 0
I20260811 22:21:48.308857 7938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab.
Entry for principal oryx/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab.
I20260811 22:21:48.348014 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41983
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab
--principal=oryx/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:48.468200 7958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:48.468494 7958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:48.468540 7958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:48.472747 7958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:48.472824 7958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:48.472878 7958 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:48.472920 7958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:48.478494 7958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab
--principal=oryx/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:41983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:48.479753 7958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:48.480813 7958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:48.490720 7972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.490809 7976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.490720 7971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:48.491087 7958 server_base.cc:1034] running on GCE node
I20260811 22:21:48.491593 7958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:48.492306 7958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:48.493479 7958 hybrid_clock.cc:648] HybridClock initialized: now 1786486908493451 us; error 38 us; skew 500 ppm
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:48.496585 7958 init.cc:377] Logged in from keytab as oryx/127.26.125.1@KRBTEST.COM (short username oryx)
I20260811 22:21:48.497893 7958 webserver.cc:460] Webserver started at http://127.26.125.1:45813/ using document root <none> and password file <none>
I20260811 22:21:48.498553 7958 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:48.498632 7958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:48.498891 7958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:48.501212 7958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "9713cd8a592b4cd3ab98e54f8667af95"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "e914848f6b61d947360026b43a9c8f52"
server_key_iv: "d1a0dfdaa70cad6e080916d5610805dc"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.501837 7958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "9713cd8a592b4cd3ab98e54f8667af95"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "e914848f6b61d947360026b43a9c8f52"
server_key_iv: "d1a0dfdaa70cad6e080916d5610805dc"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.506070 7958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:21:48.508896 7988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.510459 7958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:21:48.510604 7958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "9713cd8a592b4cd3ab98e54f8667af95"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "e914848f6b61d947360026b43a9c8f52"
server_key_iv: "d1a0dfdaa70cad6e080916d5610805dc"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.510726 7958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:48.524574 7958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:48.527881 7958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:48.528067 7958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:48.529592 7958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:48.529647 7958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.529690 7958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:48.529721 7958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.539526 7958 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:43867
I20260811 22:21:48.539673 8191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:43867 every 8 connection(s)
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:48.540776 7958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:48.544780 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.530627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:60965 (local address 127.26.125.62:41983)
0811 22:21:48.530794 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.530799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.531769 (+ 970us) server_negotiation.cc:407] Connection header received
0811 22:21:48.533098 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.533102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.533165 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.533220 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.535183 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.535980 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.537379 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.537633 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.537727 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.540753 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.540757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.540759 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.540789 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.542509 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.543125 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.543129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.543130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.543218 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.543588 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.543591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.543592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.543802 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.543890 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.544445 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.544602 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:21:48.545670 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.531106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41983 (local address 127.26.125.1:60965)
0811 22:21:48.531603 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:48.531659 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:48.532805 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:48.533650 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:48.533660 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:48.534187 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:48.534975 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.535000 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.536151 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.536155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.537225 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.537240 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.537803 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.537807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.538015 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.538678 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:48.538697 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:48.540509 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:48.542639 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.542646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.542664 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.543008 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.543303 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.543306 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.543308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.543483 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.543909 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:48.543916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:48.544188 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 22:21:48.544358 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 22:21:48.544740 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":107,"threads_started":1}
I20260811 22:21:48.546435 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 7958
I20260811 22:21:48.546586 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:48.546969 27124 external_mini_cluster.cc:1428] Setting key c33eaea5414bf36d1c2a0c9e10b6a578
I20260811 22:21:48.547246 8192 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41983
I20260811 22:21:48.547499 8192 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:48.548152 8192 heartbeater.cc:511] Master 127.26.125.62:41983 requested a full tablet report, sending...
I20260811 22:21:48.549942 7838 ts_manager.cc:194] Registered new tserver with Master: 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867)
I20260811 22:21:48.551545 7838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:60965
I20260811 22:21:48.557341 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.549867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:48595 (local address 127.26.125.62:41983)
0811 22:21:48.549994 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.549999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.550320 (+ 321us) server_negotiation.cc:407] Connection header received
0811 22:21:48.550443 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.550446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.550499 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.550574 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.551122 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.551631 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.552551 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.552784 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.552880 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.554145 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.554148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.554150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.554174 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.555560 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.556079 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.556081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.556082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.556156 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.556442 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.556445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.556446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.556705 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.556782 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.557090 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.557201 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:48.557416 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.549867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41983 (local address 127.26.125.1:48595)
0811 22:21:48.550231 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:48.550246 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:48.550348 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:48.550603 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:48.550605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:48.550842 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:48.551015 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.551024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.551781 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.551785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.552445 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.552456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.552898 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.552902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.553004 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.553565 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:48.553579 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:48.554023 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:48.555674 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.555678 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.555680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.555987 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.556234 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.556237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.556239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.556342 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.556798 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:48.556801 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:48.556918 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 22:21:48.557034 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:21:48.557202 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for oryx/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab.
Entry for principal oryx/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab.
I20260811 22:21:48.581472 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41983
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab
--principal=oryx/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:48.699594 8208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:48.699892 8208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:48.699946 8208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:48.703994 8208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:48.704070 8208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:48.704111 8208 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:48.704144 8208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:48.709676 8208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab
--principal=oryx/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:41983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:48.710909 8208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:48.712005 8208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:48.722023 8218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.722031 8215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.722033 8214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:48.722216 8208 server_base.cc:1034] running on GCE node
I20260811 22:21:48.722683 8208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:48.723417 8208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:48.724594 8208 hybrid_clock.cc:648] HybridClock initialized: now 1786486908724562 us; error 43 us; skew 500 ppm
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:48.727578 8208 init.cc:377] Logged in from keytab as oryx/127.26.125.2@KRBTEST.COM (short username oryx)
I20260811 22:21:48.728490 8208 webserver.cc:460] Webserver started at http://127.26.125.2:43787/ using document root <none> and password file <none>
I20260811 22:21:48.729086 8208 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:48.729151 8208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:48.729329 8208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:48.731505 8208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "b344007aa3690a345c79d214f3719c1b"
server_key_iv: "b9cd179430e9d5ba3f68e3c7a9220ae2"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.732048 8208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "b344007aa3690a345c79d214f3719c1b"
server_key_iv: "b9cd179430e9d5ba3f68e3c7a9220ae2"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.735898 8208 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:48.738356 8234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.739758 8208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:21:48.739873 8208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "b344007aa3690a345c79d214f3719c1b"
server_key_iv: "b9cd179430e9d5ba3f68e3c7a9220ae2"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.739987 8208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:48.753984 8208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:48.757375 8208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:48.757563 8208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:48.759218 8208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:48.759279 8208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.759347 8208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:48.759377 8208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:48.770262 8208 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:37269
I20260811 22:21:48.770378 8444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:37269 every 8 connection(s)
I20260811 22:21:48.771284 8208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:48.772970 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.759927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:37539 (local address 127.26.125.62:41983)
0811 22:21:48.760083 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.760089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.761093 (+ 1004us) server_negotiation.cc:407] Connection header received
0811 22:21:48.762367 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.762370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.762505 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.762592 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.763924 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.764650 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.765795 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.766125 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.766223 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.769071 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.769075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.769078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.769108 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.770611 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.771182 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.771186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.771188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.771271 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.771648 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.771652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.771654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.771903 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.772036 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.772707 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.772844 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:21:48.773875 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.760362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41983 (local address 127.26.125.2:37539)
0811 22:21:48.760892 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:48.760956 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:48.762114 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:48.762837 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:48.762844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:48.763247 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:48.763750 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.763764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.764773 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.764777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.765676 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.765686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.766293 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.766297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.766464 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.767111 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:48.767131 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:48.768881 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:48.770716 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.770722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.770740 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.771089 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.771352 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.771356 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.771358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.771535 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.772066 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:48.772072 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:48.772386 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 22:21:48.772631 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 22:21:48.772935 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":107,"threads_started":1}
I20260811 22:21:48.775243 8445 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41983
I20260811 22:21:48.775405 8445 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:48.775971 8445 heartbeater.cc:511] Master 127.26.125.62:41983 requested a full tablet report, sending...
I20260811 22:21:48.777063 7837 ts_manager.cc:194] Registered new tserver with Master: cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269)
I20260811 22:21:48.777784 7837 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:37539
I20260811 22:21:48.779901 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 8208
I20260811 22:21:48.780018 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:48.780311 27124 external_mini_cluster.cc:1428] Setting key 996e2a508943201e7653f83ed95bb631
I20260811 22:21:48.784376 8453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.777277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41983 (local address 127.26.125.2:34001)
0811 22:21:48.777560 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:48.777573 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:48.777658 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:48.777870 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:48.777873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:48.778087 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:48.778218 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.778226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.779027 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.779031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.779599 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.779631 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.780147 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.780151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.780241 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.780768 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:48.780783 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:48.781191 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:48.782815 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.782818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.782820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.783128 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.783370 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.783373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.783375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.783473 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.783843 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:48.783846 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:48.783958 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 22:21:48.784076 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:21:48.784217 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":127,"threads_started":1}
I20260811 22:21:48.784417 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.777331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:34001 (local address 127.26.125.62:41983)
0811 22:21:48.777453 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.777458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.777635 (+ 177us) server_negotiation.cc:407] Connection header received
0811 22:21:48.777721 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.777724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.777771 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.777865 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.778312 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.778896 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.779754 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.779994 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.780096 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.781320 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.781324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.781327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.781354 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.782703 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.783214 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.783217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.783218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.783296 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.783546 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.783548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.783550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.783765 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.783840 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.784143 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.784264 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for oryx/127.26.125.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab.
Entry for principal oryx/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab.
I20260811 22:21:48.810185 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41983
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab
--principal=oryx/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:48.929148 8457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:48.929428 8457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:48.929474 8457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:48.933620 8457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:48.933686 8457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:48.933733 8457 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:21:48.933779 8457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:48.939242 8457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab
--principal=oryx/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.125.62:41983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:48.940366 8457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:48.941483 8457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:48.951830 8466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.951833 8469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.951905 8467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:48.952253 8457 server_base.cc:1034] running on GCE node
I20260811 22:21:48.952803 8457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:48.953523 8457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:48.954716 8457 hybrid_clock.cc:648] HybridClock initialized: now 1786486908954690 us; error 36 us; skew 500 ppm
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:48.957803 8457 init.cc:377] Logged in from keytab as oryx/127.26.125.3@KRBTEST.COM (short username oryx)
I20260811 22:21:48.958871 8457 webserver.cc:460] Webserver started at http://127.26.125.3:46241/ using document root <none> and password file <none>
I20260811 22:21:48.959481 8457 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:48.959553 8457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:48.959764 8457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:48.961920 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "447b1f636ce34600b158a0358d8cc0cf"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "628de71aa7cb9613049ffc0164f3d985"
server_key_iv: "7f7e6f31abc730fad8fea696d90a4ad2"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.962530 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "447b1f636ce34600b158a0358d8cc0cf"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "628de71aa7cb9613049ffc0164f3d985"
server_key_iv: "7f7e6f31abc730fad8fea696d90a4ad2"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.966583 8457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:21:48.969214 8479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.970556 8457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 22:21:48.970675 8457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "447b1f636ce34600b158a0358d8cc0cf"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-5bn2"
server_key: "628de71aa7cb9613049ffc0164f3d985"
server_key_iv: "7f7e6f31abc730fad8fea696d90a4ad2"
server_key_version: "encryptionkey@0"
I20260811 22:21:48.970778 8457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:48.981212 8457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:48.984529 8457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:48.984719 8457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:48.986227 8457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:48.986281 8457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.986336 8457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:48.986363 8457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:21:48 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:48.997185 8457 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:43409
I20260811 22:21:48.997215 8679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:43409 every 8 connection(s)
I20260811 22:21:48.998226 8457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:49.000778 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.987179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:59939 (local address 127.26.125.62:41983)
0811 22:21:48.987297 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.987301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.988426 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 22:21:48.989586 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.989591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.989658 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.989716 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.991353 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.991910 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.993124 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.993475 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.993615 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.996520 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.996525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.996527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.996562 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.998347 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.999040 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.999044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.999046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.999159 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.999567 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.999569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.999571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.999785 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.999868 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.000520 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.000648 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:21:49.001667 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.987762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41983 (local address 127.26.125.3:59939)
0811 22:21:48.988273 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:48.988317 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:48.989310 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:48.990186 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:48.990194 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:48.990610 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:48.991195 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.991211 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.992044 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.992047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.992954 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.992967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.993689 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.993695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.993907 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.994633 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:48.994651 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:48.996289 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:48.998473 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.998487 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.998506 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.998900 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.999262 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.999264 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.999266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.999424 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.999919 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:48.999926 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:49.000214 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.000389 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.000761 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":153,"threads_started":1}
I20260811 22:21:49.003166 8680 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41983
I20260811 22:21:49.003345 8680 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:49.004016 8680 heartbeater.cc:511] Master 127.26.125.62:41983 requested a full tablet report, sending...
I20260811 22:21:49.005317 7838 ts_manager.cc:194] Registered new tserver with Master: 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409)
I20260811 22:21:49.006134 7838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:59939
I20260811 22:21:49.008574 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 8457
I20260811 22:21:49.008718 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:49.009074 27124 external_mini_cluster.cc:1428] Setting key 48a7cd308de1bc392eb5d62b4ed9f3af
I20260811 22:21:49.011790 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:21:49.013509 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.005554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:51911 (local address 127.26.125.62:41983)
0811 22:21:49.005672 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.005676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.005816 (+ 140us) server_negotiation.cc:407] Connection header received
0811 22:21:49.005918 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.005920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.005963 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.006040 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:49.006580 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.007309 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.008433 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.008756 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.008876 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.010044 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:49.010048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:49.010051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:49.010090 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:49.011816 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:49.012320 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:49.012323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:49.012325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:49.012421 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:49.012692 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:49.012694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:49.012695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:49.012881 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:49.012980 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.013248 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.013355 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:21:49.013549 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.005379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41983 (local address 127.26.125.3:51911)
0811 22:21:49.005696 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.005712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.005828 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.006069 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.006072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.006320 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:49.006466 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.006473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.007464 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.007467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.008271 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.008285 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.008879 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.008882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.008961 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.009530 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:49.009545 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:49.009925 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:49.011955 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:49.011958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:49.011961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:49.012201 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:49.012509 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:49.012512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:49.012514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:49.012614 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:49.012988 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:49.012991 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:49.013100 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.013203 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.013357 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":115,"threads_started":1}
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:21:49.039487 7917 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:49.032799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57810 (local address 127.26.125.62:41983)
0811 22:21:49.032944 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.032948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.032962 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:49.033009 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.033011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.033053 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.033124 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:49.034013 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.034586 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.035605 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.035827 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.035908 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.039337 (+ 3429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57810: BlockingRecv error: recv got EOF from 127.0.0.1:57810 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:49.049192 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.041060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57824 (local address 127.26.125.62:41983)
0811 22:21:49.041189 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.041193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.041296 (+ 103us) server_negotiation.cc:407] Connection header received
0811 22:21:49.041449 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.041452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.041510 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.041591 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:49.042219 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.042776 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.043624 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.043840 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.043923 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.046282 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:49.046286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:49.046288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:49.046315 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:49.047700 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:49.048078 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:49.048081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:49.048082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:49.048169 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:49.048352 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:49.048354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:49.048355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:49.048597 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:49.048706 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.048965 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.049065 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:21:49.052297 7838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:49.055897 7838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:49.072291 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.065617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:37269 (local address 127.0.0.1:53230)
0811 22:21:49.066075 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.066131 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.066263 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.068091 (+ 1828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.068094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.068112 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.068213 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.068219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.069839 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.069843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.070668 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.070680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.071639 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.071643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.071790 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.071837 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.071972 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.072115 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260811 22:21:49.072314 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.066466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:43867 (local address 127.0.0.1:35704)
0811 22:21:49.066773 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.066785 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.066871 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.067799 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.067802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.067822 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.067982 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.067989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.069023 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.069026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.070624 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.070637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.071835 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.071839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.072072 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.072095 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.072160 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.072220 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":187,"thread_start_us":102,"threads_started":1}
I20260811 22:21:49.072753 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.065660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43409 (local address 127.0.0.1:43614)
0811 22:21:49.066585 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.066600 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.066689 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.067633 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.067639 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.067664 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.067868 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.067878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.069201 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.069204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.070624 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.070637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.071835 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.071839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.071958 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.072131 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.072198 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.072280 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":204,"thread_start_us":107,"threads_started":1}
I20260811 22:21:49.072968 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.066668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.26.125.3:43409)
0811 22:21:49.067118 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.067128 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.067149 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:21:49.067217 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.067223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.067458 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.067624 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.067989 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.069086 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.070745 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.071633 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.072042 (+ 409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.072196 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.072251 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.072828 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":132,"threads_started":1}
I20260811 22:21:49.073314 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.067110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.26.125.1:43867)
0811 22:21:49.067384 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.067398 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.067420 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:21:49.067492 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.067497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.067682 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.067818 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.068107 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.068909 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.070946 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.071705 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.071885 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.072059 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.072992 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.073179 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260811 22:21:49.074000 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.065777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53230 (local address 127.26.125.2:37269)
0811 22:21:49.067480 (+ 1703us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.067489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.067510 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:21:49.067581 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.067587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.067786 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.067947 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.068343 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.069716 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.070792 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.071511 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.071622 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.071900 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.072415 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.073108 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1605,"thread_start_us":115,"threads_started":1}
I20260811 22:21:49.076335 8571 tablet_service.cc:1467] Processing CreateTablet for tablet f5c983f0f149451aa167eb41d3e3410a (DEFAULT_TABLE table=test-table [id=86f62cbe484e44c685617a06faeb9b2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.076337 8060 tablet_service.cc:1467] Processing CreateTablet for tablet f5c983f0f149451aa167eb41d3e3410a (DEFAULT_TABLE table=test-table [id=86f62cbe484e44c685617a06faeb9b2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.076993 8325 tablet_service.cc:1467] Processing CreateTablet for tablet f5c983f0f149451aa167eb41d3e3410a (DEFAULT_TABLE table=test-table [id=86f62cbe484e44c685617a06faeb9b2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.077463 8571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5c983f0f149451aa167eb41d3e3410a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.077486 8060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5c983f0f149451aa167eb41d3e3410a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.077992 8325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5c983f0f149451aa167eb41d3e3410a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.084267 8733 tablet_bootstrap.cc:492] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: Bootstrap starting.
I20260811 22:21:49.086042 8737 tablet_bootstrap.cc:492] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: Bootstrap starting.
I20260811 22:21:49.086719 8735 tablet_bootstrap.cc:492] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: Bootstrap starting.
I20260811 22:21:49.087395 8733 tablet_bootstrap.cc:654] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.088199 8737 tablet_bootstrap.cc:654] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.088428 8733 log.cc:824] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:49.089001 8735 tablet_bootstrap.cc:654] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.089023 8737 log.cc:824] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:49.089794 8735 log.cc:824] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:49.090830 8733 tablet_bootstrap.cc:492] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: No bootstrap required, opened a new log
I20260811 22:21:49.091081 8733 ts_tablet_manager.cc:1397] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260811 22:21:49.091174 8737 tablet_bootstrap.cc:492] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: No bootstrap required, opened a new log
I20260811 22:21:49.091373 8737 ts_tablet_manager.cc:1397] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:21:49.092396 8735 tablet_bootstrap.cc:492] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: No bootstrap required, opened a new log
I20260811 22:21:49.092594 8735 ts_tablet_manager.cc:1397] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:21:49.094574 8737 raft_consensus.cc:348] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.094836 8737 raft_consensus.cc:374] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.094918 8737 raft_consensus.cc:729] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc12c3cfd7c04f4a9e2908a08b50d40d, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.095371 8737 consensus_queue.cc:260] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.095321 8733 raft_consensus.cc:348] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.095602 8733 raft_consensus.cc:374] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.095660 8733 raft_consensus.cc:729] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447b1f636ce34600b158a0358d8cc0cf, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.096191 8445 heartbeater.cc:503] Master 127.26.125.62:41983 was elected leader, sending a full tablet report...
I20260811 22:21:49.096483 8737 ts_tablet_manager.cc:1428] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260811 22:21:49.096777 8733 consensus_queue.cc:260] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.097050 8735 raft_consensus.cc:348] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.097298 8735 raft_consensus.cc:374] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.097666 8735 raft_consensus.cc:729] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9713cd8a592b4cd3ab98e54f8667af95, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.097801 8733 ts_tablet_manager.cc:1428] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260811 22:21:49.098193 8680 heartbeater.cc:503] Master 127.26.125.62:41983 was elected leader, sending a full tablet report...
I20260811 22:21:49.098140 8735 consensus_queue.cc:260] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.098803 8192 heartbeater.cc:503] Master 127.26.125.62:41983 was elected leader, sending a full tablet report...
I20260811 22:21:49.100518 8735 ts_tablet_manager.cc:1428] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260811 22:21:49.168686 8745 raft_consensus.cc:482] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:49.168932 8745 raft_consensus.cc:504] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.170253 8745 leader_election.cc:288] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269), 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409)
I20260811 22:21:49.176033 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.170734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:40829 (local address 127.26.125.3:43409)
0811 22:21:49.170913 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.170918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.171099 (+ 181us) server_negotiation.cc:407] Connection header received
0811 22:21:49.171273 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.171276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.171332 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.171419 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.171886 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.172721 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.174686 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.175379 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.175500 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.175559 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.175821 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.175890 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:21:49.176033 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.170748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:42867 (local address 127.26.125.2:37269)
0811 22:21:49.170986 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.170991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.171154 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:21:49.171306 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.171310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.171353 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.171419 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.171886 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.172721 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.174686 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.175366 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.175547 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.175593 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.175815 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.175891 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260811 22:21:49.176146 8746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.170576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43409 (local address 127.26.125.1:40829)
0811 22:21:49.170985 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.171000 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.171164 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.171465 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.171468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.171517 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.171742 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.171749 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.172888 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.172892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.174535 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.174545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.175508 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.175512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.175663 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.175713 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.175800 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.175957 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":345,"spinlock_wait_cycles":8064,"thread_start_us":131,"threads_started":1}
I20260811 22:21:49.176154 8747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.170576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:37269 (local address 127.26.125.1:42867)
0811 22:21:49.171078 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.171090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.171206 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.171469 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.171471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.171515 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.171742 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.171749 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.172888 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.172892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.174531 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.174545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.175497 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.175501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.175659 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.175714 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.175793 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.175955 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":450,"spinlock_wait_cycles":78848,"thread_start_us":88,"threads_started":1}
I20260811 22:21:49.176833 8327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5c983f0f149451aa167eb41d3e3410a" candidate_uuid: "9713cd8a592b4cd3ab98e54f8667af95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" is_pre_election: true
I20260811 22:21:49.177040 8610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5c983f0f149451aa167eb41d3e3410a" candidate_uuid: "9713cd8a592b4cd3ab98e54f8667af95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447b1f636ce34600b158a0358d8cc0cf" is_pre_election: true
I20260811 22:21:49.177225 8327 raft_consensus.cc:2454] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9713cd8a592b4cd3ab98e54f8667af95 in term 0.
I20260811 22:21:49.177300 8610 raft_consensus.cc:2454] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9713cd8a592b4cd3ab98e54f8667af95 in term 0.
I20260811 22:21:49.177767 7989 leader_election.cc:302] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 447b1f636ce34600b158a0358d8cc0cf, 9713cd8a592b4cd3ab98e54f8667af95, cc12c3cfd7c04f4a9e2908a08b50d40d; no voters:
I20260811 22:21:49.178141 8745 raft_consensus.cc:2781] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:49.178243 8745 raft_consensus.cc:482] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:49.178298 8745 raft_consensus.cc:3037] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.179472 8745 raft_consensus.cc:504] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.179806 8745 leader_election.cc:288] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [CANDIDATE]: Term 1 election: Requested vote from peers cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269), 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409)
I20260811 22:21:49.180176 8610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5c983f0f149451aa167eb41d3e3410a" candidate_uuid: "9713cd8a592b4cd3ab98e54f8667af95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447b1f636ce34600b158a0358d8cc0cf"
I20260811 22:21:49.180179 8327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5c983f0f149451aa167eb41d3e3410a" candidate_uuid: "9713cd8a592b4cd3ab98e54f8667af95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d"
I20260811 22:21:49.180307 8610 raft_consensus.cc:3037] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.180307 8327 raft_consensus.cc:3037] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.181644 8327 raft_consensus.cc:2454] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9713cd8a592b4cd3ab98e54f8667af95 in term 1.
I20260811 22:21:49.181850 8610 raft_consensus.cc:2454] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9713cd8a592b4cd3ab98e54f8667af95 in term 1.
I20260811 22:21:49.181967 7989 leader_election.cc:302] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9713cd8a592b4cd3ab98e54f8667af95, cc12c3cfd7c04f4a9e2908a08b50d40d; no voters:
I20260811 22:21:49.182173 8745 raft_consensus.cc:2781] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:49.182389 8745 raft_consensus.cc:686] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 LEADER]: Becoming Leader. State: Replica: 9713cd8a592b4cd3ab98e54f8667af95, State: Running, Role: LEADER
I20260811 22:21:49.182724 8745 consensus_queue.cc:237] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } }
I20260811 22:21:49.186242 7838 catalog_manager.cc:5495] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1). New cstate: current_term: 1 leader_uuid: "9713cd8a592b4cd3ab98e54f8667af95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } health_report { overall_health: HEALTHY } } }
I20260811 22:21:49.206645 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.202685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.26.125.1:43867)
0811 22:21:49.202911 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.202915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.202931 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:21:49.203189 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.203192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.203282 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.203382 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:49.203744 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.204322 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.205190 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.205399 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.205514 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.205737 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.206235 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.206323 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.206388 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.206486 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 22:21:49.213454 8610 raft_consensus.cc:1261] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Refusing update from remote peer 9713cd8a592b4cd3ab98e54f8667af95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.213696 8327 raft_consensus.cc:1261] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Refusing update from remote peer 9713cd8a592b4cd3ab98e54f8667af95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.214313 8749 consensus_queue.cc:1035] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.214550 8745 consensus_queue.cc:1035] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.223042 8756 mvcc.cc:205] Tried to move back new op lower bound from 7317450380129792000 to 7317450380015390720. Current Snapshot: MvccSnapshot[applied={T|T < 7317450380129792000}]
I20260811 22:21:49.223155 8754 mvcc.cc:205] Tried to move back new op lower bound from 7317450380129792000 to 7317450380015390720. Current Snapshot: MvccSnapshot[applied={T|T < 7317450380129792000}]
I20260811 22:21:49.232366 7838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:49.232576 7838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:49.235325 7838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: Sending DeleteTablet for 3 replicas of tablet f5c983f0f149451aa167eb41d3e3410a
I20260811 22:21:49.236279 8325 tablet_service.cc:1514] Processing DeleteTablet for tablet f5c983f0f149451aa167eb41d3e3410a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:53230
I20260811 22:21:49.236289 8571 tablet_service.cc:1514] Processing DeleteTablet for tablet f5c983f0f149451aa167eb41d3e3410a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:43614
I20260811 22:21:49.236888 8774 tablet_replica.cc:330] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: stopping tablet replica
I20260811 22:21:49.236984 7838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:21:49.237197 8060 tablet_service.cc:1514] Processing DeleteTablet for tablet f5c983f0f149451aa167eb41d3e3410a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35704
I20260811 22:21:49.237290 8774 raft_consensus.cc:2229] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 22:21:49.237452 7838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:49.237733 8774 raft_consensus.cc:2258] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.238009 8776 tablet_replica.cc:330] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: stopping tablet replica
I20260811 22:21:49.238140 8773 tablet_replica.cc:330] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: stopping tablet replica
I20260811 22:21:49.238440 8773 raft_consensus.cc:2229] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:49.238435 8776 raft_consensus.cc:2229] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:49.238749 8773 raft_consensus.cc:2258] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.240298 8776 raft_consensus.cc:2258] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.240969 8774 ts_tablet_manager.cc:1905] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.241758 8776 ts_tablet_manager.cc:1905] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.243175 8571 tablet_service.cc:1467] Processing CreateTablet for tablet c2025beb806d429eace6fc3f241c4a54 (DEFAULT_TABLE table=test-table [id=957a08a81398493f8caf4d46e2a830f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.243175 8325 tablet_service.cc:1467] Processing CreateTablet for tablet c2025beb806d429eace6fc3f241c4a54 (DEFAULT_TABLE table=test-table [id=957a08a81398493f8caf4d46e2a830f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.243458 8571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2025beb806d429eace6fc3f241c4a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.243458 8325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2025beb806d429eace6fc3f241c4a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.245853 8737 tablet_bootstrap.cc:492] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Bootstrap starting.
I20260811 22:21:49.246104 8060 tablet_service.cc:1467] Processing CreateTablet for tablet c2025beb806d429eace6fc3f241c4a54 (DEFAULT_TABLE table=test-table [id=957a08a81398493f8caf4d46e2a830f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.246294 8733 tablet_bootstrap.cc:492] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: Bootstrap starting.
I20260811 22:21:49.246392 8060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2025beb806d429eace6fc3f241c4a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.246944 8737 tablet_bootstrap.cc:654] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.247301 8733 tablet_bootstrap.cc:654] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.249150 8735 tablet_bootstrap.cc:492] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: Bootstrap starting.
I20260811 22:21:49.249699 8773 ts_tablet_manager.cc:1905] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.250201 8735 tablet_bootstrap.cc:654] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.251521 8776 ts_tablet_manager.cc:1918] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:49.251639 8776 log.cc:1196] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/f5c983f0f149451aa167eb41d3e3410a
I20260811 22:21:49.251957 8776 ts_tablet_manager.cc:1939] T f5c983f0f149451aa167eb41d3e3410a P 9713cd8a592b4cd3ab98e54f8667af95: Deleting consensus metadata
I20260811 22:21:49.253278 7814 catalog_manager.cc:4841] TS 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867): tablet f5c983f0f149451aa167eb41d3e3410a (table test-table [id=86f62cbe484e44c685617a06faeb9b2d]) successfully deleted
I20260811 22:21:49.254699 8774 ts_tablet_manager.cc:1918] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:49.254818 8774 log.cc:1196] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/f5c983f0f149451aa167eb41d3e3410a
I20260811 22:21:49.255132 8774 ts_tablet_manager.cc:1939] T f5c983f0f149451aa167eb41d3e3410a P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting consensus metadata
I20260811 22:21:49.256996 8773 ts_tablet_manager.cc:1918] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:49.257100 8773 log.cc:1196] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/f5c983f0f149451aa167eb41d3e3410a
I20260811 22:21:49.257385 8773 ts_tablet_manager.cc:1939] T f5c983f0f149451aa167eb41d3e3410a P 447b1f636ce34600b158a0358d8cc0cf: Deleting consensus metadata
I20260811 22:21:49.258634 7815 catalog_manager.cc:4841] TS 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409): tablet f5c983f0f149451aa167eb41d3e3410a (table test-table [id=86f62cbe484e44c685617a06faeb9b2d]) successfully deleted
I20260811 22:21:49.259307 8733 tablet_bootstrap.cc:492] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: No bootstrap required, opened a new log
I20260811 22:21:49.259404 8733 ts_tablet_manager.cc:1397] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260811 22:21:49.259932 8733 raft_consensus.cc:348] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.260070 8733 raft_consensus.cc:374] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.260106 8733 raft_consensus.cc:729] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447b1f636ce34600b158a0358d8cc0cf, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.260226 8733 consensus_queue.cc:260] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.260514 8733 ts_tablet_manager.cc:1428] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 22:21:49.260908 8737 tablet_bootstrap.cc:492] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: No bootstrap required, opened a new log
I20260811 22:21:49.260998 8737 ts_tablet_manager.cc:1397] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20260811 22:21:49.261438 8737 raft_consensus.cc:348] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.261620 8737 raft_consensus.cc:374] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.261654 8737 raft_consensus.cc:729] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc12c3cfd7c04f4a9e2908a08b50d40d, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.261705 7814 catalog_manager.cc:4841] TS cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269): tablet f5c983f0f149451aa167eb41d3e3410a (table test-table [id=86f62cbe484e44c685617a06faeb9b2d]) successfully deleted
I20260811 22:21:49.261778 8737 consensus_queue.cc:260] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.262035 8737 ts_tablet_manager.cc:1428] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:21:49.265148 8735 tablet_bootstrap.cc:492] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: No bootstrap required, opened a new log
I20260811 22:21:49.265251 8735 ts_tablet_manager.cc:1397] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.001s
I20260811 22:21:49.265724 8735 raft_consensus.cc:348] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.265839 8735 raft_consensus.cc:374] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.265861 8735 raft_consensus.cc:729] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9713cd8a592b4cd3ab98e54f8667af95, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.266018 8735 consensus_queue.cc:260] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.266280 8735 ts_tablet_manager.cc:1428] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260811 22:21:49.272552 8447 tablet.cc:2369] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:21:49.292514 8194 tablet.cc:2369] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:49.348397 8744 raft_consensus.cc:482] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:49.348570 8744 raft_consensus.cc:504] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.349866 8744 leader_election.cc:288] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867), cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269)
I20260811 22:21:49.354786 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.350808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:60571 (local address 127.26.125.2:37269)
0811 22:21:49.350996 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.351000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.351015 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:49.351124 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.351127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.351173 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.351236 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.351569 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.352375 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.353725 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.354139 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.354259 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.354324 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.354545 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.354643 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:21:49.354784 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.350128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:43867 (local address 127.26.125.3:36531)
0811 22:21:49.350702 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.350718 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.350819 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.351194 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.351197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.351217 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.351358 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.351366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.352146 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.352150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.353333 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.353345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.354401 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.354404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.354509 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.354533 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.354592 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.354651 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":1}
I20260811 22:21:49.354810 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.350454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:37269 (local address 127.26.125.3:60571)
0811 22:21:49.350890 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.350906 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.351018 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.351311 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.351314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.351333 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.351453 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.351459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.352501 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.352505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.353597 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.353605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.354282 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.354287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.354388 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.354414 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.354494 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.354594 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260811 22:21:49.355273 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.350261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:36531 (local address 127.26.125.1:43867)
0811 22:21:49.350387 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.350392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.350908 (+ 516us) server_negotiation.cc:407] Connection header received
0811 22:21:49.351031 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.351035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.351089 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.351178 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.351491 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.352026 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.353500 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.354178 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.354313 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.354387 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.355055 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.355149 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:21:49.355819 8103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2025beb806d429eace6fc3f241c4a54" candidate_uuid: "447b1f636ce34600b158a0358d8cc0cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9713cd8a592b4cd3ab98e54f8667af95" is_pre_election: true
I20260811 22:21:49.355980 8327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2025beb806d429eace6fc3f241c4a54" candidate_uuid: "447b1f636ce34600b158a0358d8cc0cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" is_pre_election: true
I20260811 22:21:49.356129 8103 raft_consensus.cc:2454] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 447b1f636ce34600b158a0358d8cc0cf in term 0.
I20260811 22:21:49.356154 8327 raft_consensus.cc:2454] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 447b1f636ce34600b158a0358d8cc0cf in term 0.
I20260811 22:21:49.356564 8481 leader_election.cc:302] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447b1f636ce34600b158a0358d8cc0cf, cc12c3cfd7c04f4a9e2908a08b50d40d; no voters:
I20260811 22:21:49.356845 8744 raft_consensus.cc:2781] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:49.356913 8744 raft_consensus.cc:482] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:49.356951 8744 raft_consensus.cc:3037] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.358044 8744 raft_consensus.cc:504] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.358418 8744 leader_election.cc:288] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [CANDIDATE]: Term 1 election: Requested vote from peers 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867), cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269)
I20260811 22:21:49.358948 8103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2025beb806d429eace6fc3f241c4a54" candidate_uuid: "447b1f636ce34600b158a0358d8cc0cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9713cd8a592b4cd3ab98e54f8667af95"
I20260811 22:21:49.358966 8327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2025beb806d429eace6fc3f241c4a54" candidate_uuid: "447b1f636ce34600b158a0358d8cc0cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d"
I20260811 22:21:49.359076 8103 raft_consensus.cc:3037] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.359109 8327 raft_consensus.cc:3037] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.360163 8103 raft_consensus.cc:2454] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 447b1f636ce34600b158a0358d8cc0cf in term 1.
I20260811 22:21:49.360487 8327 raft_consensus.cc:2454] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 447b1f636ce34600b158a0358d8cc0cf in term 1.
I20260811 22:21:49.360502 8481 leader_election.cc:302] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447b1f636ce34600b158a0358d8cc0cf, 9713cd8a592b4cd3ab98e54f8667af95; no voters:
I20260811 22:21:49.360683 8744 raft_consensus.cc:2781] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:49.360925 8744 raft_consensus.cc:686] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 LEADER]: Becoming Leader. State: Replica: 447b1f636ce34600b158a0358d8cc0cf, State: Running, Role: LEADER
I20260811 22:21:49.361202 8744 consensus_queue.cc:237] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.363914 7836 catalog_manager.cc:5495] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3). New cstate: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } health_report { overall_health: HEALTHY } } }
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.62@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:49.413209 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.404921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57832 (local address 127.26.125.62:41983)
0811 22:21:49.405065 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.405068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.405265 (+ 197us) server_negotiation.cc:407] Connection header received
0811 22:21:49.405407 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.405410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.405452 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.405545 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:49.406216 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.406751 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.407571 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.407783 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.407854 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.410190 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:49.410193 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:49.410195 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:49.410218 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:49.411679 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:49.412122 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:49.412124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:49.412125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:49.412197 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:49.412389 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:49.412391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:49.412392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:49.412602 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:49.412674 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.412921 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.413075 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":58}
I20260811 22:21:49.413210 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.404859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41983 (local address 127.0.0.1:57832)
0811 22:21:49.405182 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.405197 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.405308 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.405571 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.405573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.406014 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:49.406118 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.406124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.406902 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.406905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.407472 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.407481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.407863 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.407866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.407941 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.408541 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:49.408554 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:49.410036 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:49.411789 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:49.411799 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:49.411806 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:49.412042 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:49.412256 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:49.412258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:49.412259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:49.412331 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:49.412676 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:49.412681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:49.412794 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.412880 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.413016 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":4,"thread_start_us":111,"threads_started":1}
I20260811 22:21:49.417457 7836 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:57832:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:21:49.418287 7836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:49.423174 8060 tablet_service.cc:1467] Processing CreateTablet for tablet 0abdf3290cfb47db8ef14118266a56dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c05a0b8ee43c4f4687df6bd5b02b7ca2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:49.423223 8571 tablet_service.cc:1467] Processing CreateTablet for tablet 0abdf3290cfb47db8ef14118266a56dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c05a0b8ee43c4f4687df6bd5b02b7ca2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:49.423223 8325 tablet_service.cc:1467] Processing CreateTablet for tablet 0abdf3290cfb47db8ef14118266a56dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c05a0b8ee43c4f4687df6bd5b02b7ca2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:21:49.423620 8571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0abdf3290cfb47db8ef14118266a56dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.423620 8325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0abdf3290cfb47db8ef14118266a56dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.423624 8060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0abdf3290cfb47db8ef14118266a56dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.425957 8735 tablet_bootstrap.cc:492] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: Bootstrap starting.
I20260811 22:21:49.426589 8737 tablet_bootstrap.cc:492] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: Bootstrap starting.
I20260811 22:21:49.426782 8735 tablet_bootstrap.cc:654] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.426986 8733 tablet_bootstrap.cc:492] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: Bootstrap starting.
I20260811 22:21:49.427498 8737 tablet_bootstrap.cc:654] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.427901 8733 tablet_bootstrap.cc:654] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.428282 8735 tablet_bootstrap.cc:492] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: No bootstrap required, opened a new log
I20260811 22:21:49.428372 8735 ts_tablet_manager.cc:1397] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:21:49.428639 8737 tablet_bootstrap.cc:492] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: No bootstrap required, opened a new log
I20260811 22:21:49.428727 8737 ts_tablet_manager.cc:1397] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:21:49.428803 8735 raft_consensus.cc:348] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.428921 8735 raft_consensus.cc:374] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.428953 8735 raft_consensus.cc:729] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9713cd8a592b4cd3ab98e54f8667af95, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.429071 8735 consensus_queue.cc:260] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.429154 8737 raft_consensus.cc:348] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.429265 8737 raft_consensus.cc:374] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.429297 8737 raft_consensus.cc:729] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc12c3cfd7c04f4a9e2908a08b50d40d, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.429337 8735 ts_tablet_manager.cc:1428] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:49.429410 8737 consensus_queue.cc:260] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.429648 8737 ts_tablet_manager.cc:1428] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:49.429824 8733 tablet_bootstrap.cc:492] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: No bootstrap required, opened a new log
I20260811 22:21:49.429899 8733 ts_tablet_manager.cc:1397] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 22:21:49.429845 8742 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.429993 8742 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:49.429973 8749 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.430104 8749 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:49.430328 8733 raft_consensus.cc:348] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.430449 8733 raft_consensus.cc:374] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.430485 8733 raft_consensus.cc:729] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447b1f636ce34600b158a0358d8cc0cf, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.430615 8733 consensus_queue.cc:260] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.430909 8733 ts_tablet_manager.cc:1428] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:49.430878 8782 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.431039 8782 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:49.443820 8782 raft_consensus.cc:482] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:49.443995 8782 raft_consensus.cc:504] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.444442 8782 leader_election.cc:288] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867), cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269)
I20260811 22:21:49.444911 8103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0abdf3290cfb47db8ef14118266a56dd" candidate_uuid: "447b1f636ce34600b158a0358d8cc0cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9713cd8a592b4cd3ab98e54f8667af95" is_pre_election: true
I20260811 22:21:49.444952 8327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0abdf3290cfb47db8ef14118266a56dd" candidate_uuid: "447b1f636ce34600b158a0358d8cc0cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" is_pre_election: true
I20260811 22:21:49.445073 8103 raft_consensus.cc:2454] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 447b1f636ce34600b158a0358d8cc0cf in term 0.
I20260811 22:21:49.445111 8327 raft_consensus.cc:2454] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 447b1f636ce34600b158a0358d8cc0cf in term 0.
I20260811 22:21:49.445389 8481 leader_election.cc:302] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447b1f636ce34600b158a0358d8cc0cf, 9713cd8a592b4cd3ab98e54f8667af95; no voters:
I20260811 22:21:49.445636 8782 raft_consensus.cc:2781] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:49.445701 8782 raft_consensus.cc:482] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:49.445727 8782 raft_consensus.cc:3037] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.446918 8782 raft_consensus.cc:504] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.447312 8782 leader_election.cc:288] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [CANDIDATE]: Term 1 election: Requested vote from peers 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867), cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269)
I20260811 22:21:49.447713 8103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0abdf3290cfb47db8ef14118266a56dd" candidate_uuid: "447b1f636ce34600b158a0358d8cc0cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9713cd8a592b4cd3ab98e54f8667af95"
I20260811 22:21:49.447751 8327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0abdf3290cfb47db8ef14118266a56dd" candidate_uuid: "447b1f636ce34600b158a0358d8cc0cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d"
I20260811 22:21:49.447844 8103 raft_consensus.cc:3037] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.447867 8327 raft_consensus.cc:3037] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.448863 8327 raft_consensus.cc:2454] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 447b1f636ce34600b158a0358d8cc0cf in term 1.
I20260811 22:21:49.448936 8103 raft_consensus.cc:2454] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 447b1f636ce34600b158a0358d8cc0cf in term 1.
I20260811 22:21:49.449206 8481 leader_election.cc:302] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447b1f636ce34600b158a0358d8cc0cf, cc12c3cfd7c04f4a9e2908a08b50d40d; no voters:
I20260811 22:21:49.449397 8782 raft_consensus.cc:2781] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:49.449493 8782 raft_consensus.cc:686] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [term 1 LEADER]: Becoming Leader. State: Replica: 447b1f636ce34600b158a0358d8cc0cf, State: Running, Role: LEADER
I20260811 22:21:49.449636 8782 consensus_queue.cc:237] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.450439 8744 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: TxnStatusTablet state changed. Reason: New leader 447b1f636ce34600b158a0358d8cc0cf. Latest consensus state: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.450549 8744 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:49.450982 8805 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:21:49.451156 8805 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:21:49.451536 7837 catalog_manager.cc:5495] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3). New cstate: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } health_report { overall_health: HEALTHY } } }
I20260811 22:21:49.467293 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.463350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43409 (local address 127.0.0.1:43630)
0811 22:21:49.463609 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.463625 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.463753 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.464103 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.464120 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.464129 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:49.464238 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.464244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.464963 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.464967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.465576 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.465585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.466036 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.466040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.466135 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.466185 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.466970 (+ 785us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.466991 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.467067 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.467157 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260811 22:21:49.467293 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.463481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43630 (local address 127.26.125.3:43409)
0811 22:21:49.463626 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.463630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.463672 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:21:49.463898 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.463902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.463966 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.464061 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:49.464342 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.464852 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.465688 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.465920 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.466010 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.466326 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.466816 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.466992 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.467078 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.467138 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:21:49.470921 8327 raft_consensus.cc:1261] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Refusing update from remote peer 447b1f636ce34600b158a0358d8cc0cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.471199 8103 raft_consensus.cc:1261] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Refusing update from remote peer 447b1f636ce34600b158a0358d8cc0cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.471606 8782 consensus_queue.cc:1035] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.471823 8744 consensus_queue.cc:1035] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.473483 8742 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: TxnStatusTablet state changed. Reason: New leader 447b1f636ce34600b158a0358d8cc0cf. Latest consensus state: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.473620 8742 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:49.474701 8749 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: TxnStatusTablet state changed. Reason: New leader 447b1f636ce34600b158a0358d8cc0cf. Latest consensus state: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.474815 8749 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:49.477039 8782 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.477157 8782 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:49.477488 8752 mvcc.cc:205] Tried to move back new op lower bound from 7317450381186891776 to 7317450381107163136. Current Snapshot: MvccSnapshot[applied={T|T < 7317450381186891776}]
I20260811 22:21:49.477663 8749 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.477751 8749 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P 9713cd8a592b4cd3ab98e54f8667af95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:49.479720 8744 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.479831 8744 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P 447b1f636ce34600b158a0358d8cc0cf: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:21:49.481200 8742 tablet_replica.cc:439] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "447b1f636ce34600b158a0358d8cc0cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } } }
I20260811 22:21:49.481320 8742 tablet_replica.cc:442] T 0abdf3290cfb47db8ef14118266a56dd P cc12c3cfd7c04f4a9e2908a08b50d40d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:21:49.486789 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.483260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.26.125.3:43409)
0811 22:21:49.483378 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.483382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.483474 (+ 92us) server_negotiation.cc:407] Connection header received
0811 22:21:49.483655 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.483658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.483705 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.483774 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:49.484061 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.484605 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.485811 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.486038 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.486125 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.486389 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.486465 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.486525 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.486606 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.486655 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:21:49.497970 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.494912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:50049 (local address 127.26.125.3:43409)
0811 22:21:49.495016 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.495020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.495034 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:49.495170 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.495173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.495212 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.495268 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:49.495499 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.495978 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.496835 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.497042 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.497126 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.497414 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.497540 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.497634 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.497748 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.497791 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":40}
I20260811 22:21:49.497970 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.494799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43409 (local address 127.26.125.3:50049)
0811 22:21:49.494975 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.494990 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.495084 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.495283 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.495285 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.495291 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:49.495411 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.495416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.496093 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.496097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.496727 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.496736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.497136 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.497139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.497219 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.497268 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.497636 (+ 368us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.497642 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.497707 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.497797 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":2}
W20260811 22:21:49.499648 8681 tablet.cc:2369] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:49.505424 8327 raft_consensus.cc:1261] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Refusing update from remote peer 447b1f636ce34600b158a0358d8cc0cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.505501 8103 raft_consensus.cc:1261] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Refusing update from remote peer 447b1f636ce34600b158a0358d8cc0cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.505980 8744 consensus_queue.cc:1035] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.506165 8782 consensus_queue.cc:1035] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.573186 7836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:49.573356 7836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:49.575623 7836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: Sending DeleteTablet for 3 replicas of tablet c2025beb806d429eace6fc3f241c4a54
I20260811 22:21:49.576359 8060 tablet_service.cc:1514] Processing DeleteTablet for tablet c2025beb806d429eace6fc3f241c4a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35704
I20260811 22:21:49.576613 8776 tablet_replica.cc:330] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: stopping tablet replica
I20260811 22:21:49.576678 8325 tablet_service.cc:1514] Processing DeleteTablet for tablet c2025beb806d429eace6fc3f241c4a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:53230
I20260811 22:21:49.576763 8776 raft_consensus.cc:2229] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:49.576864 8774 tablet_replica.cc:330] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: stopping tablet replica
I20260811 22:21:49.576889 8776 raft_consensus.cc:2258] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.576972 8774 raft_consensus.cc:2229] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:49.576982 8570 tablet_service.cc:1514] Processing DeleteTablet for tablet c2025beb806d429eace6fc3f241c4a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:43614
I20260811 22:21:49.577095 8774 raft_consensus.cc:2258] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.577186 8773 tablet_replica.cc:330] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: stopping tablet replica
I20260811 22:21:49.577299 8773 raft_consensus.cc:2229] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:49.577615 8773 raft_consensus.cc:2258] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.578039 8774 ts_tablet_manager.cc:1905] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.578372 8773 ts_tablet_manager.cc:1905] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.581050 8773 ts_tablet_manager.cc:1918] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:49.581050 8774 ts_tablet_manager.cc:1918] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:49.581138 8773 log.cc:1196] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/c2025beb806d429eace6fc3f241c4a54
I20260811 22:21:49.581138 8774 log.cc:1196] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/c2025beb806d429eace6fc3f241c4a54
I20260811 22:21:49.581395 8773 ts_tablet_manager.cc:1939] T c2025beb806d429eace6fc3f241c4a54 P 447b1f636ce34600b158a0358d8cc0cf: Deleting consensus metadata
I20260811 22:21:49.581394 8774 ts_tablet_manager.cc:1939] T c2025beb806d429eace6fc3f241c4a54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting consensus metadata
I20260811 22:21:49.582170 7815 catalog_manager.cc:4841] TS 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409): tablet c2025beb806d429eace6fc3f241c4a54 (table test-table [id=957a08a81398493f8caf4d46e2a830f1]) successfully deleted
I20260811 22:21:49.582509 7814 catalog_manager.cc:4841] TS cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269): tablet c2025beb806d429eace6fc3f241c4a54 (table test-table [id=957a08a81398493f8caf4d46e2a830f1]) successfully deleted
I20260811 22:21:49.582731 8776 ts_tablet_manager.cc:1905] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.585094 8776 ts_tablet_manager.cc:1918] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:49.585171 8776 log.cc:1196] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/c2025beb806d429eace6fc3f241c4a54
I20260811 22:21:49.585382 8776 ts_tablet_manager.cc:1939] T c2025beb806d429eace6fc3f241c4a54 P 9713cd8a592b4cd3ab98e54f8667af95: Deleting consensus metadata
I20260811 22:21:49.586154 7814 catalog_manager.cc:4841] TS 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867): tablet c2025beb806d429eace6fc3f241c4a54 (table test-table [id=957a08a81398493f8caf4d46e2a830f1]) successfully deleted
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:21:49.610453 7917 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:49.604156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57836 (local address 127.26.125.62:41983)
0811 22:21:49.604293 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.604298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.604467 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:21:49.604626 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.604629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.604682 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.604752 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:49.605518 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.606053 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.607001 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.607236 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.607330 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.610320 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57836: BlockingRecv error: recv got EOF from 127.0.0.1:57836 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 22:21:49 dist-test-slave-5bn2 krb5kdc[7760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM
I20260811 22:21:49.620903 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.612059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57850 (local address 127.26.125.62:41983)
0811 22:21:49.612180 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.612185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.612275 (+ 90us) server_negotiation.cc:407] Connection header received
0811 22:21:49.612408 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.612411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.612457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.612520 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:49.613269 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.614080 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.615073 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.615450 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.615567 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.617866 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:49.617870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:49.617873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:49.617905 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:49.619399 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:49.619781 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:49.619784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:49.619786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:49.619870 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:49.620074 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:49.620076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:49.620078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:49.620286 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:49.620407 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.620603 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.620721 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:49.623963 7836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:49.624452 7836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:49.629467 8060 tablet_service.cc:1467] Processing CreateTablet for tablet caa520929584496fbd486a9f45f22c54 (DEFAULT_TABLE table=test-table [id=eb904f131ed24971977754cc83dcb349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.629467 8325 tablet_service.cc:1467] Processing CreateTablet for tablet caa520929584496fbd486a9f45f22c54 (DEFAULT_TABLE table=test-table [id=eb904f131ed24971977754cc83dcb349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.629648 8570 tablet_service.cc:1467] Processing CreateTablet for tablet caa520929584496fbd486a9f45f22c54 (DEFAULT_TABLE table=test-table [id=eb904f131ed24971977754cc83dcb349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.629812 8325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa520929584496fbd486a9f45f22c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.629812 8060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa520929584496fbd486a9f45f22c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.629961 8570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa520929584496fbd486a9f45f22c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.632346 8737 tablet_bootstrap.cc:492] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Bootstrap starting.
I20260811 22:21:49.632586 8735 tablet_bootstrap.cc:492] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: Bootstrap starting.
I20260811 22:21:49.633271 8737 tablet_bootstrap.cc:654] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.633456 8735 tablet_bootstrap.cc:654] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.634034 8733 tablet_bootstrap.cc:492] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: Bootstrap starting.
I20260811 22:21:49.634690 8737 tablet_bootstrap.cc:492] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: No bootstrap required, opened a new log
I20260811 22:21:49.634780 8737 ts_tablet_manager.cc:1397] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:21:49.634941 8733 tablet_bootstrap.cc:654] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.635268 8737 raft_consensus.cc:348] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.635407 8737 raft_consensus.cc:374] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.635440 8737 raft_consensus.cc:729] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc12c3cfd7c04f4a9e2908a08b50d40d, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.635581 8737 consensus_queue.cc:260] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.635846 8737 ts_tablet_manager.cc:1428] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:49.636713 8733 tablet_bootstrap.cc:492] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: No bootstrap required, opened a new log
I20260811 22:21:49.636797 8733 ts_tablet_manager.cc:1397] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 22:21:49.637200 8733 raft_consensus.cc:348] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.637313 8733 raft_consensus.cc:374] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.637342 8733 raft_consensus.cc:729] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447b1f636ce34600b158a0358d8cc0cf, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.637461 8733 consensus_queue.cc:260] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.637718 8733 ts_tablet_manager.cc:1428] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:21:49.638612 8735 tablet_bootstrap.cc:492] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: No bootstrap required, opened a new log
I20260811 22:21:49.638705 8735 ts_tablet_manager.cc:1397] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260811 22:21:49.639132 8735 raft_consensus.cc:348] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.639252 8735 raft_consensus.cc:374] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.639282 8735 raft_consensus.cc:729] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9713cd8a592b4cd3ab98e54f8667af95, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.639403 8735 consensus_queue.cc:260] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.639678 8735 ts_tablet_manager.cc:1428] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:49.641330 8742 raft_consensus.cc:482] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:49.641439 8742 raft_consensus.cc:504] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.642743 8742 leader_election.cc:288] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867), 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409)
I20260811 22:21:49.647485 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.643148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:40887 (local address 127.26.125.3:43409)
0811 22:21:49.643406 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.643410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.643639 (+ 229us) server_negotiation.cc:407] Connection header received
0811 22:21:49.643856 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.643859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.643906 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.643953 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.644425 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.644980 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.646205 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.646645 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.646764 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.646825 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.647234 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.647325 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260811 22:21:49.647969 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.642960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43409 (local address 127.26.125.2:40887)
0811 22:21:49.643545 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.643560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.643740 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.644095 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.644098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.644119 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.644299 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.644304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.645104 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.645107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.646080 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.646092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.646960 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.646964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.647080 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.647110 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.647723 (+ 613us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.647816 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":42,"spinlock_wait_cycles":33792,"thread_start_us":90,"threads_started":1}
I20260811 22:21:49.648389 8610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa520929584496fbd486a9f45f22c54" candidate_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447b1f636ce34600b158a0358d8cc0cf" is_pre_election: true
I20260811 22:21:49.648541 8610 raft_consensus.cc:2454] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc12c3cfd7c04f4a9e2908a08b50d40d in term 0.
I20260811 22:21:49.648975 8238 leader_election.cc:302] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447b1f636ce34600b158a0358d8cc0cf, cc12c3cfd7c04f4a9e2908a08b50d40d; no voters:
I20260811 22:21:49.649096 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.643149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:57991 (local address 127.26.125.1:43867)
0811 22:21:49.643269 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.643274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.643650 (+ 376us) server_negotiation.cc:407] Connection header received
0811 22:21:49.643856 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.643859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.643918 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.643960 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:49.644459 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.645246 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.647285 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.647926 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.648026 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.648078 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.648876 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.648979 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:21:49.649309 8742 raft_consensus.cc:2781] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:49.649365 8742 raft_consensus.cc:482] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:49.649397 8742 raft_consensus.cc:3037] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.649495 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.643256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:43867 (local address 127.26.125.2:57991)
0811 22:21:49.643582 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.643595 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.643740 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.644104 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.644106 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.644128 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:49.644294 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.644303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.645794 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.645798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.647133 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.647145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.648558 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.648563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.648732 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.648758 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.648834 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.648921 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":21,"spinlock_wait_cycles":15872,"thread_start_us":77,"threads_started":1}
I20260811 22:21:49.650075 8106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa520929584496fbd486a9f45f22c54" candidate_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9713cd8a592b4cd3ab98e54f8667af95" is_pre_election: true
I20260811 22:21:49.650236 8106 raft_consensus.cc:2454] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc12c3cfd7c04f4a9e2908a08b50d40d in term 0.
I20260811 22:21:49.650527 8742 raft_consensus.cc:504] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.650947 8742 leader_election.cc:288] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [CANDIDATE]: Term 1 election: Requested vote from peers 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867), 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409)
I20260811 22:21:49.651273 8106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa520929584496fbd486a9f45f22c54" candidate_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9713cd8a592b4cd3ab98e54f8667af95"
I20260811 22:21:49.651384 8106 raft_consensus.cc:3037] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.651409 8610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa520929584496fbd486a9f45f22c54" candidate_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447b1f636ce34600b158a0358d8cc0cf"
I20260811 22:21:49.651532 8610 raft_consensus.cc:3037] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.652513 8610 raft_consensus.cc:2454] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc12c3cfd7c04f4a9e2908a08b50d40d in term 1.
I20260811 22:21:49.652582 8106 raft_consensus.cc:2454] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc12c3cfd7c04f4a9e2908a08b50d40d in term 1.
I20260811 22:21:49.652840 8238 leader_election.cc:302] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447b1f636ce34600b158a0358d8cc0cf, cc12c3cfd7c04f4a9e2908a08b50d40d; no voters:
I20260811 22:21:49.653019 8742 raft_consensus.cc:2781] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:49.653259 8742 raft_consensus.cc:686] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 LEADER]: Becoming Leader. State: Replica: cc12c3cfd7c04f4a9e2908a08b50d40d, State: Running, Role: LEADER
I20260811 22:21:49.653534 8742 consensus_queue.cc:237] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.656164 7836 catalog_manager.cc:5495] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d reported cstate change: term changed from 0 to 1, leader changed from <none> to cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2). New cstate: current_term: 1 leader_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } health_report { overall_health: UNKNOWN } } }
I20260811 22:21:49.671088 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.667166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53234 (local address 127.26.125.2:37269)
0811 22:21:49.667293 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.667296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.667309 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:21:49.667451 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.667453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.667547 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.667641 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:49.667877 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.668369 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.669552 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.669760 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.669829 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.670119 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.670671 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.670760 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.670879 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.670944 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:21:49.673899 8106 raft_consensus.cc:1261] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Refusing update from remote peer cc12c3cfd7c04f4a9e2908a08b50d40d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.674007 8610 raft_consensus.cc:1261] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Refusing update from remote peer cc12c3cfd7c04f4a9e2908a08b50d40d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.674434 8858 consensus_queue.cc:1035] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.674710 8742 consensus_queue.cc:1035] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [LEADER]: Connected to new peer: Peer: permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.688997 7838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:49.689194 7838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:49.691762 7838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: Sending DeleteTablet for 3 replicas of tablet caa520929584496fbd486a9f45f22c54
I20260811 22:21:49.692528 8060 tablet_service.cc:1514] Processing DeleteTablet for tablet caa520929584496fbd486a9f45f22c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35704
I20260811 22:21:49.692759 8570 tablet_service.cc:1514] Processing DeleteTablet for tablet caa520929584496fbd486a9f45f22c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:43614
I20260811 22:21:49.692938 8773 tablet_replica.cc:330] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: stopping tablet replica
I20260811 22:21:49.692919 8776 tablet_replica.cc:330] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: stopping tablet replica
I20260811 22:21:49.693066 8773 raft_consensus.cc:2229] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:49.693069 8776 raft_consensus.cc:2229] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:49.693207 8776 raft_consensus.cc:2258] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.693208 8773 raft_consensus.cc:2258] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.693617 8325 tablet_service.cc:1514] Processing DeleteTablet for tablet caa520929584496fbd486a9f45f22c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:53230
I20260811 22:21:49.693810 8774 tablet_replica.cc:330] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: stopping tablet replica
I20260811 22:21:49.693913 8774 raft_consensus.cc:2229] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:49.693961 8773 ts_tablet_manager.cc:1905] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.694273 8774 raft_consensus.cc:2258] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.695650 8776 ts_tablet_manager.cc:1905] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.695984 7838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:49.696444 7838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:49.696707 8773 ts_tablet_manager.cc:1918] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:49.696781 8773 log.cc:1196] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/caa520929584496fbd486a9f45f22c54
I20260811 22:21:49.697031 8773 ts_tablet_manager.cc:1939] T caa520929584496fbd486a9f45f22c54 P 447b1f636ce34600b158a0358d8cc0cf: Deleting consensus metadata
I20260811 22:21:49.698304 8776 ts_tablet_manager.cc:1918] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:49.698299 7815 catalog_manager.cc:4841] TS 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409): tablet caa520929584496fbd486a9f45f22c54 (table test-table [id=eb904f131ed24971977754cc83dcb349]) successfully deleted
I20260811 22:21:49.698385 8776 log.cc:1196] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/caa520929584496fbd486a9f45f22c54
I20260811 22:21:49.698645 8776 ts_tablet_manager.cc:1939] T caa520929584496fbd486a9f45f22c54 P 9713cd8a592b4cd3ab98e54f8667af95: Deleting consensus metadata
I20260811 22:21:49.699410 7814 catalog_manager.cc:4841] TS 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867): tablet caa520929584496fbd486a9f45f22c54 (table test-table [id=eb904f131ed24971977754cc83dcb349]) successfully deleted
I20260811 22:21:49.700935 8774 ts_tablet_manager.cc:1905] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.703425 8774 ts_tablet_manager.cc:1918] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:49.703559 8774 log.cc:1196] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/caa520929584496fbd486a9f45f22c54
I20260811 22:21:49.704257 8060 tablet_service.cc:1467] Processing CreateTablet for tablet 19f14bea93a1479490d317e5f75f75cb (DEFAULT_TABLE table=test-table [id=8f4877ed64c641da840127317ee4506d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.704257 8325 tablet_service.cc:1467] Processing CreateTablet for tablet 19f14bea93a1479490d317e5f75f75cb (DEFAULT_TABLE table=test-table [id=8f4877ed64c641da840127317ee4506d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.704514 8060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f14bea93a1479490d317e5f75f75cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.704530 8325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f14bea93a1479490d317e5f75f75cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.706729 8735 tablet_bootstrap.cc:492] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: Bootstrap starting.
I20260811 22:21:49.706765 8737 tablet_bootstrap.cc:492] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: Bootstrap starting.
I20260811 22:21:49.707718 8735 tablet_bootstrap.cc:654] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.707782 8737 tablet_bootstrap.cc:654] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.703809 8774 ts_tablet_manager.cc:1939] T caa520929584496fbd486a9f45f22c54 P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting consensus metadata
I20260811 22:21:49.708930 8570 tablet_service.cc:1467] Processing CreateTablet for tablet 19f14bea93a1479490d317e5f75f75cb (DEFAULT_TABLE table=test-table [id=8f4877ed64c641da840127317ee4506d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:49.709231 8570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f14bea93a1479490d317e5f75f75cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:49.709601 7814 catalog_manager.cc:4841] TS cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269): tablet caa520929584496fbd486a9f45f22c54 (table test-table [id=eb904f131ed24971977754cc83dcb349]) successfully deleted
I20260811 22:21:49.711460 8737 tablet_bootstrap.cc:492] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: No bootstrap required, opened a new log
I20260811 22:21:49.711558 8737 ts_tablet_manager.cc:1397] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:21:49.712054 8737 raft_consensus.cc:348] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.712174 8737 raft_consensus.cc:374] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.712204 8737 raft_consensus.cc:729] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc12c3cfd7c04f4a9e2908a08b50d40d, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.712322 8737 consensus_queue.cc:260] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.712392 8735 tablet_bootstrap.cc:492] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: No bootstrap required, opened a new log
I20260811 22:21:49.712481 8735 ts_tablet_manager.cc:1397] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 22:21:49.712600 8737 ts_tablet_manager.cc:1428] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:49.712880 8735 raft_consensus.cc:348] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.712986 8735 raft_consensus.cc:374] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.713025 8735 raft_consensus.cc:729] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9713cd8a592b4cd3ab98e54f8667af95, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.713145 8735 consensus_queue.cc:260] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.713395 8735 ts_tablet_manager.cc:1428] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:49.713990 8733 tablet_bootstrap.cc:492] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: Bootstrap starting.
I20260811 22:21:49.714970 8733 tablet_bootstrap.cc:654] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:49.718146 8733 tablet_bootstrap.cc:492] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: No bootstrap required, opened a new log
I20260811 22:21:49.718256 8733 ts_tablet_manager.cc:1397] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260811 22:21:49.718688 8733 raft_consensus.cc:348] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.718820 8733 raft_consensus.cc:374] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:49.718925 8733 raft_consensus.cc:729] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447b1f636ce34600b158a0358d8cc0cf, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.719080 8733 consensus_queue.cc:260] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.719355 8733 ts_tablet_manager.cc:1428] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:21:49.853085 8749 raft_consensus.cc:482] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:49.853271 8749 raft_consensus.cc:504] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.853735 8749 leader_election.cc:288] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269), 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409)
I20260811 22:21:49.854259 8327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f14bea93a1479490d317e5f75f75cb" candidate_uuid: "9713cd8a592b4cd3ab98e54f8667af95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" is_pre_election: true
I20260811 22:21:49.854259 8610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f14bea93a1479490d317e5f75f75cb" candidate_uuid: "9713cd8a592b4cd3ab98e54f8667af95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447b1f636ce34600b158a0358d8cc0cf" is_pre_election: true
I20260811 22:21:49.854432 8610 raft_consensus.cc:2454] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9713cd8a592b4cd3ab98e54f8667af95 in term 0.
I20260811 22:21:49.854460 8327 raft_consensus.cc:2454] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9713cd8a592b4cd3ab98e54f8667af95 in term 0.
I20260811 22:21:49.854799 7989 leader_election.cc:302] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9713cd8a592b4cd3ab98e54f8667af95, cc12c3cfd7c04f4a9e2908a08b50d40d; no voters:
I20260811 22:21:49.855046 8749 raft_consensus.cc:2781] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:49.855123 8749 raft_consensus.cc:482] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:49.855161 8749 raft_consensus.cc:3037] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.856426 8749 raft_consensus.cc:504] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.856807 8749 leader_election.cc:288] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [CANDIDATE]: Term 1 election: Requested vote from peers cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269), 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409)
I20260811 22:21:49.857223 8610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f14bea93a1479490d317e5f75f75cb" candidate_uuid: "9713cd8a592b4cd3ab98e54f8667af95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447b1f636ce34600b158a0358d8cc0cf"
I20260811 22:21:49.857223 8327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f14bea93a1479490d317e5f75f75cb" candidate_uuid: "9713cd8a592b4cd3ab98e54f8667af95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d"
I20260811 22:21:49.857362 8610 raft_consensus.cc:3037] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.857362 8327 raft_consensus.cc:3037] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:49.858577 8327 raft_consensus.cc:2454] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9713cd8a592b4cd3ab98e54f8667af95 in term 1.
I20260811 22:21:49.858577 8610 raft_consensus.cc:2454] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9713cd8a592b4cd3ab98e54f8667af95 in term 1.
I20260811 22:21:49.858937 7991 leader_election.cc:302] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447b1f636ce34600b158a0358d8cc0cf, 9713cd8a592b4cd3ab98e54f8667af95; no voters:
I20260811 22:21:49.859181 8749 raft_consensus.cc:2781] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:49.859350 8749 raft_consensus.cc:686] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 LEADER]: Becoming Leader. State: Replica: 9713cd8a592b4cd3ab98e54f8667af95, State: Running, Role: LEADER
I20260811 22:21:49.859514 8749 consensus_queue.cc:237] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } }
I20260811 22:21:49.861217 7837 catalog_manager.cc:5495] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1). New cstate: current_term: 1 leader_uuid: "9713cd8a592b4cd3ab98e54f8667af95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9713cd8a592b4cd3ab98e54f8667af95" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 } health_report { overall_health: UNKNOWN } } }
I20260811 22:21:49.877168 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.874027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35714 (local address 127.26.125.1:43867)
0811 22:21:49.874149 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.874154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.874167 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:21:49.874336 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.874338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.874384 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.874447 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:49.874656 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.875300 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.876219 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.876479 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.876568 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.876754 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.876828 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.876872 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.876941 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.876992 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:21:49.888443 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.885150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:60891 (local address 127.26.125.3:43409)
0811 22:21:49.885292 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.885297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.885475 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:21:49.885588 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.885592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.885642 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.885703 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:49.885922 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.886436 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.887382 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.887624 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.887713 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.888003 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.888084 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.888141 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.888228 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.888282 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:49.888458 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.885036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:43409 (local address 127.26.125.1:60891)
0811 22:21:49.885361 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.885375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.885499 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.885718 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.885720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.885728 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:49.885829 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.885834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.886540 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.886544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.887259 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.887269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.887726 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.887729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.887818 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.887868 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.888154 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.888161 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.888217 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.888302 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":128,"threads_started":1}
I20260811 22:21:49.894377 8610 raft_consensus.cc:1261] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Refusing update from remote peer 9713cd8a592b4cd3ab98e54f8667af95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.894377 8327 raft_consensus.cc:1261] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Refusing update from remote peer 9713cd8a592b4cd3ab98e54f8667af95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:49.894745 8880 consensus_queue.cc:1035] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc12c3cfd7c04f4a9e2908a08b50d40d" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.894961 8749 consensus_queue.cc:1035] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "447b1f636ce34600b158a0358d8cc0cf" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 43409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:49.911677 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.908565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:52105 (local address 127.26.125.1:43867)
0811 22:21:49.908680 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:49.908685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:49.908758 (+ 73us) server_negotiation.cc:407] Connection header received
0811 22:21:49.908900 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:49.908903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:49.908986 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:49.909058 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:49.909278 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.909781 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.910632 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.910892 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.910989 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.911224 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.911303 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.911359 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:49.911477 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 22:21:49.911542 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 22:21:49.911679 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:49.908490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:43867 (local address 127.26.125.3:52105)
0811 22:21:49.908703 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:49.908719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:49.908807 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:49.909074 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:49.909077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:49.909086 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:21:49.909184 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.909190 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.909881 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.909885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.910523 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:49.910531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:49.910997 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:49.911001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:49.911092 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:49.911121 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:21:49.911376 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:21:49.911383 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:21:49.911451 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:21:49.911532 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 22:21:49.933364 7837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:49.933555 7837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:49.935731 7837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43cac8993a264ab8a76b1e83267f3c5c: Sending DeleteTablet for 3 replicas of tablet 19f14bea93a1479490d317e5f75f75cb
I20260811 22:21:49.936414 8570 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f14bea93a1479490d317e5f75f75cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:43614
I20260811 22:21:49.936434 8060 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f14bea93a1479490d317e5f75f75cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35704
I20260811 22:21:49.936421 8325 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f14bea93a1479490d317e5f75f75cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:49 UTC) from {username='oryx', principal='oryx/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:53230
I20260811 22:21:49.936648 8773 tablet_replica.cc:330] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: stopping tablet replica
I20260811 22:21:49.936683 8774 tablet_replica.cc:330] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: stopping tablet replica
I20260811 22:21:49.936775 8773 raft_consensus.cc:2229] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:49.936797 8774 raft_consensus.cc:2229] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:49.936892 8773 raft_consensus.cc:2258] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.936898 8774 raft_consensus.cc:2258] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.937062 8776 tablet_replica.cc:330] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: stopping tablet replica
I20260811 22:21:49.937181 8776 raft_consensus.cc:2229] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:49.937382 8776 raft_consensus.cc:2258] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:49.937656 8774 ts_tablet_manager.cc:1905] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.937902 8773 ts_tablet_manager.cc:1905] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.938155 8776 ts_tablet_manager.cc:1905] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:49.940104 8773 ts_tablet_manager.cc:1918] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:49.940187 8773 log.cc:1196] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/19f14bea93a1479490d317e5f75f75cb
I20260811 22:21:49.940358 8774 ts_tablet_manager.cc:1918] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:49.940433 8773 ts_tablet_manager.cc:1939] T 19f14bea93a1479490d317e5f75f75cb P 447b1f636ce34600b158a0358d8cc0cf: Deleting consensus metadata
I20260811 22:21:49.940466 8774 log.cc:1196] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/19f14bea93a1479490d317e5f75f75cb
I20260811 22:21:49.940598 8776 ts_tablet_manager.cc:1918] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:21:49.940675 8776 log.cc:1196] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/19f14bea93a1479490d317e5f75f75cb
I20260811 22:21:49.940737 8774 ts_tablet_manager.cc:1939] T 19f14bea93a1479490d317e5f75f75cb P cc12c3cfd7c04f4a9e2908a08b50d40d: Deleting consensus metadata
I20260811 22:21:49.940908 8776 ts_tablet_manager.cc:1939] T 19f14bea93a1479490d317e5f75f75cb P 9713cd8a592b4cd3ab98e54f8667af95: Deleting consensus metadata
I20260811 22:21:49.941197 7815 catalog_manager.cc:4841] TS 447b1f636ce34600b158a0358d8cc0cf (127.26.125.3:43409): tablet 19f14bea93a1479490d317e5f75f75cb (table test-table [id=8f4877ed64c641da840127317ee4506d]) successfully deleted
I20260811 22:21:49.941457 7814 catalog_manager.cc:4841] TS cc12c3cfd7c04f4a9e2908a08b50d40d (127.26.125.2:37269): tablet 19f14bea93a1479490d317e5f75f75cb (table test-table [id=8f4877ed64c641da840127317ee4506d]) successfully deleted
I20260811 22:21:49.941636 7814 catalog_manager.cc:4841] TS 9713cd8a592b4cd3ab98e54f8667af95 (127.26.125.1:43867): tablet 19f14bea93a1479490d317e5f75f75cb (table test-table [id=8f4877ed64c641da840127317ee4506d]) successfully deleted
I20260811 22:21:49.941856 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 7958
I20260811 22:21:49.964319 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 8208
I20260811 22:21:49.987154 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 8457
I20260811 22:21:50.009487 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 7787
2026-08-11T22:21:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3284 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:50 dist-test-slave-5bn2 krb5kdc[8900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:50 dist-test-slave-5bn2 krb5kdc[8900](info): set up 2 sockets
Aug 11 22:21:50 dist-test-slave-5bn2 krb5kdc[8900](info): commencing operation
krb5kdc: starting...
W20260811 22:21:51.203634 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:51 dist-test-slave-5bn2 krb5kdc[8900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:51.339603 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40613
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:51.456117 8918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:51.456406 8918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:51.456465 8918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:51.460450 8918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:51.460526 8918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:51.460553 8918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:51.460574 8918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:51.460600 8918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:51.466059 8918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:51.467311 8918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:51.468425 8918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:51.477502 8930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:51.477547 8933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:51.477591 8931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:51.477700 8918 server_base.cc:1034] running on GCE node
I20260811 22:21:51.478390 8918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:51.479604 8918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:51.480777 8918 hybrid_clock.cc:648] HybridClock initialized: now 1786486911480756 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T22:21:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1478 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:51 dist-test-slave-5bn2 krb5kdc[8943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:51 dist-test-slave-5bn2 krb5kdc[8943](info): set up 2 sockets
Aug 11 22:21:51 dist-test-slave-5bn2 krb5kdc[8943](info): commencing operation
krb5kdc: starting...
W20260811 22:21:52.689450 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:21:52 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:21:52.822029 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40321
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:52.939031 8967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:52.939337 8967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:52.939393 8967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:52.943789 8967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:52.943877 8967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:52.943903 8967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:52.943926 8967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:52.943956 8967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:52.949605 8967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:52.950793 8967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:52.951910 8967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:52.960830 8972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:52.960815 8973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:52.960911 8967 server_base.cc:1034] running on GCE node
W20260811 22:21:52.960815 8975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:52.961755 8967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:52.963030 8967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:52.964226 8967 hybrid_clock.cc:648] HybridClock initialized: now 1786486912964197 us; error 37 us; skew 500 ppm
Aug 11 22:21:52 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:52.967149 8967 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:52.968166 8967 webserver.cc:460] Webserver started at http://127.26.125.62:34737/ using document root <none> and password file <none>
I20260811 22:21:52.968781 8967 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:52.968852 8967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:52.969065 8967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:52.971284 8967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "4bae60f74770407caf4d6baffa15719f"
format_stamp: "Formatted at 2026-08-11 22:21:52 on dist-test-slave-5bn2"
server_key: "d17d74412d56dddd9a4e55c52215051b"
server_key_iv: "3d6df757f35904c7d9cba67c6baff777"
server_key_version: "encryptionkey@0"
I20260811 22:21:52.971817 8967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "4bae60f74770407caf4d6baffa15719f"
format_stamp: "Formatted at 2026-08-11 22:21:52 on dist-test-slave-5bn2"
server_key: "d17d74412d56dddd9a4e55c52215051b"
server_key_iv: "3d6df757f35904c7d9cba67c6baff777"
server_key_version: "encryptionkey@0"
I20260811 22:21:52.975589 8967 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:21:52.978017 8996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:52.979362 8967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 22:21:52.979498 8967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "4bae60f74770407caf4d6baffa15719f"
format_stamp: "Formatted at 2026-08-11 22:21:52 on dist-test-slave-5bn2"
server_key: "d17d74412d56dddd9a4e55c52215051b"
server_key_iv: "3d6df757f35904c7d9cba67c6baff777"
server_key_version: "encryptionkey@0"
I20260811 22:21:52.979595 8967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:52.997825 8967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:53.001371 8967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:53.001560 8967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:53.008980 8967 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:40321
I20260811 22:21:53.008996 9095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:40321 every 8 connection(s)
I20260811 22:21:53.010075 8967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:53.013188 9097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:53.019068 9097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Bootstrap starting.
I20260811 22:21:53.020136 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 8967
I20260811 22:21:53.020273 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:21:53.020577 27124 external_mini_cluster.cc:1428] Setting key fb575e6b077cf7f7b0647fef083f2f31
I20260811 22:21:53.021962 9097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:53.022720 9097 log.cc:824] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:53.025017 9097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: No bootstrap required, opened a new log
I20260811 22:21:53.029410 9097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } }
I20260811 22:21:53.029662 9097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:53.029721 9097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bae60f74770407caf4d6baffa15719f, State: Initialized, Role: FOLLOWER
Aug 11 22:21:53 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:53.030267 9097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } }
I20260811 22:21:53.030442 9097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:53.030517 9097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:53.030624 9097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:53.031571 9097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } }
I20260811 22:21:53.031917 9097 leader_election.cc:302] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bae60f74770407caf4d6baffa15719f; no voters:
I20260811 22:21:53.032243 9097 leader_election.cc:288] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:53.032338 9109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:53.032521 9109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 1 LEADER]: Becoming Leader. State: Replica: 4bae60f74770407caf4d6baffa15719f, State: Running, Role: LEADER
I20260811 22:21:53.032833 9109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } }
I20260811 22:21:53.033640 9097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:53.034068 9109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bae60f74770407caf4d6baffa15719f. Latest consensus state: current_term: 1 leader_uuid: "4bae60f74770407caf4d6baffa15719f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } } }
I20260811 22:21:53.034178 9109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:53.035069 9112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bae60f74770407caf4d6baffa15719f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } } }
I20260811 22:21:53.035177 9112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:53.036053 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:53.021720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.26.125.62:40321)
0811 22:21:53.022205 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:53.022222 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:53.022261 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:21:53.022988 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:53.023011 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:53.023354 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:53.023659 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:53.024569 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.025655 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.026909 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.027451 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.027577 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:53.030487 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:53.030494 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:53.030549 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:53.030588 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:53.032383 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:53.032861 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:53.032869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:53.032876 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:53.033033 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:53.033313 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:53.033316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:53.033318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:53.033832 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:53.034023 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:53.034327 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 22:21:53.035185 (+ 858us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":146,"threads_started":1}
I20260811 22:21:53.036940 9115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:53.040338 9115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:53.046042 9115 catalog_manager.cc:1347] Generated new cluster ID: 6480c044a48d459497156c13e40ea199
I20260811 22:21:53.046114 9115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:53.094955 9115 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:53.096525 9115 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:53.105665 9115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Generated new TSK 0
I20260811 22:21:53.106788 9115 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:21:53.150796 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40321
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:53.268752 9143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:53.269065 9143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:53.269129 9143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:53.273313 9143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:53.273399 9143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:53.273484 9143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:53.279050 9143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:53.280221 9143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:53.281299 9143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:53.291659 9151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:53.291677 9152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:53.291668 9158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:53.292033 9143 server_base.cc:1034] running on GCE node
I20260811 22:21:53.292497 9143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:53.293239 9143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:53.294414 9143 hybrid_clock.cc:648] HybridClock initialized: now 1786486913294387 us; error 39 us; skew 500 ppm
Aug 11 22:21:53 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486913, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:53.297591 9143 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:21:53.298640 9143 webserver.cc:460] Webserver started at http://127.26.125.1:35277/ using document root <none> and password file <none>
I20260811 22:21:53.299302 9143 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:53.299378 9143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:53.299618 9143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:53.301816 9143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "328aca06c9ec46ca921273a44bdc5033"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "7a3db83b2783c59c79dbc06060df9589"
server_key_iv: "b2fe12de6f3a2dd35d6177675cbcaa3c"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.302459 9143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "328aca06c9ec46ca921273a44bdc5033"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "7a3db83b2783c59c79dbc06060df9589"
server_key_iv: "b2fe12de6f3a2dd35d6177675cbcaa3c"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.306712 9143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:21:53.309896 9166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.311380 9143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:53.311527 9143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "328aca06c9ec46ca921273a44bdc5033"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "7a3db83b2783c59c79dbc06060df9589"
server_key_iv: "b2fe12de6f3a2dd35d6177675cbcaa3c"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.311646 9143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:53.331554 9143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:53.335117 9143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:53.335309 9143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:53.335951 9143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:53.336829 9143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:53.336880 9143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.336936 9143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:53.336966 9143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.345779 9143 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:35943
I20260811 22:21:53.345803 9402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:35943 every 8 connection(s)
I20260811 22:21:53.346828 9143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:53.349587 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 9143
I20260811 22:21:53.349735 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:53.350131 27124 external_mini_cluster.cc:1428] Setting key 501792110da9efb653f1ea4a4af5bfa3
Aug 11 22:21:53 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486913, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:21:53.363327 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:53.348895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:57105 (local address 127.26.125.62:40321)
0811 22:21:53.349036 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:53.349042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:53.349943 (+ 901us) server_negotiation.cc:407] Connection header received
0811 22:21:53.351499 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:53.351503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:53.351569 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:53.351642 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:53.353076 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.353812 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.355780 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.356045 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.356146 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:53.359365 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:53.359369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:53.359371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:53.359402 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:53.360832 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:53.361545 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:53.361547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:53.361548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:53.361633 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:53.362042 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:53.362044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:53.362045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:53.362233 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:53.362305 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:53.363057 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 22:21:53.363185 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:53.364240 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:53.349224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:57105)
0811 22:21:53.349756 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:53.349812 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:53.351265 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:53.351943 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:53.351950 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:53.352341 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:53.352892 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:53.352908 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.354006 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.354010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:53.355605 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:53.355621 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.356230 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.356235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:53.356457 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:53.357125 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:53.357152 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:53.359123 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:53.360978 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:53.360988 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:53.361011 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:53.361407 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:53.361723 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:53.361727 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:53.361729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:53.361950 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:53.362337 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:53.362345 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:53.362741 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 22:21:53.362965 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 22:21:53.363334 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:21:53.365659 9403 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40321
I20260811 22:21:53.365936 9403 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:53.366498 9403 heartbeater.cc:511] Master 127.26.125.62:40321 requested a full tablet report, sending...
I20260811 22:21:53.368281 9029 ts_manager.cc:194] Registered new tserver with Master: 328aca06c9ec46ca921273a44bdc5033 (127.26.125.1:35943)
I20260811 22:21:53.369520 9029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:57105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:21:53.382210 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40321
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:53.501844 9417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:53.502171 9417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:53.502239 9417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:53.506645 9417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:53.506742 9417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:53.506829 9417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:53.512698 9417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:53.513907 9417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:53.515074 9417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:53.525595 9428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:53.525574 9429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:53.525568 9432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:53.525641 9417 server_base.cc:1034] running on GCE node
I20260811 22:21:53.526252 9417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:53.527151 9417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:53.528352 9417 hybrid_clock.cc:648] HybridClock initialized: now 1786486913528322 us; error 45 us; skew 500 ppm
Aug 11 22:21:53 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486913, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:53.531522 9417 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:21:53.532541 9417 webserver.cc:460] Webserver started at http://127.26.125.2:33341/ using document root <none> and password file <none>
I20260811 22:21:53.533156 9417 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:53.533257 9417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:53.533581 9417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:53.536167 9417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "d25b213843f0463bb27b089d8a5754e9"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "21fa993156afe5d1f8816078eb8ddb99"
server_key_iv: "c476bc0e9e83149f1c1dd5582ca7486c"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.536839 9417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "d25b213843f0463bb27b089d8a5754e9"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "21fa993156afe5d1f8816078eb8ddb99"
server_key_iv: "c476bc0e9e83149f1c1dd5582ca7486c"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.541229 9417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:53.543977 9441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.545414 9417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:53.545549 9417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "d25b213843f0463bb27b089d8a5754e9"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "21fa993156afe5d1f8816078eb8ddb99"
server_key_iv: "c476bc0e9e83149f1c1dd5582ca7486c"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.545670 9417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:53.570107 9417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:53.573227 9417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:53.573413 9417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:53.574064 9417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:53.575076 9417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:53.575130 9417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.575189 9417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:53.575213 9417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.584285 9417 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:35759
I20260811 22:21:53.584324 9656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:35759 every 8 connection(s)
I20260811 22:21:53.585287 9417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:53.590963 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 9417
I20260811 22:21:53.591101 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:53.591405 27124 external_mini_cluster.cc:1428] Setting key 0bd0b31b7c85cffbd2ab4a52c1a7f1b3
Aug 11 22:21:53 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486913, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:53.600504 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:53.587289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:53793 (local address 127.26.125.62:40321)
0811 22:21:53.587433 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:53.587438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:53.588415 (+ 977us) server_negotiation.cc:407] Connection header received
0811 22:21:53.589765 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:53.589767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:53.589820 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:53.589897 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:53.591395 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.592089 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.593551 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.593834 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.593942 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:53.596703 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:53.596707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:53.596710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:53.596740 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:53.598219 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:53.598777 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:53.598780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:53.598781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:53.598903 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:53.599261 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:53.599264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:53.599266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:53.599510 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:53.599602 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:53.600178 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 22:21:53.600321 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:21:53.601409 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:53.587746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:53793)
0811 22:21:53.588254 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:53.588300 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:53.589555 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:53.590154 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:53.590162 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:53.590577 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:53.591227 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:53.591245 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.592227 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.592231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:53.593421 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:53.593431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.593991 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.593995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:53.594153 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:53.594789 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:53.594807 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:53.596526 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:53.598355 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:53.598361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:53.598378 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:53.598662 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:53.599006 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:53.599008 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:53.599010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:53.599168 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:53.599616 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:53.599624 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:53.599908 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 22:21:53.600120 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 22:21:53.600492 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:21:53.602645 9657 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40321
I20260811 22:21:53.602947 9657 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:53.603498 9657 heartbeater.cc:511] Master 127.26.125.62:40321 requested a full tablet report, sending...
I20260811 22:21:53.604773 9029 ts_manager.cc:194] Registered new tserver with Master: d25b213843f0463bb27b089d8a5754e9 (127.26.125.2:35759)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:21:53.605662 9029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:53793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:21:53.622313 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40321
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:53.741935 9662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:53.742234 9662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:53.742278 9662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:53.746512 9662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:53.746598 9662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:53.746662 9662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:53.752339 9662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:53.753595 9662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:53.754788 9662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:53.765813 9662 server_base.cc:1034] running on GCE node
W20260811 22:21:53.765759 9683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:53.765764 9681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:53.765764 9680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:53.766417 9662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:53.767226 9662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:53.768494 9662 hybrid_clock.cc:648] HybridClock initialized: now 1786486913768458 us; error 61 us; skew 500 ppm
Aug 11 22:21:53 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486913, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:53.771785 9662 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:21:53.772825 9662 webserver.cc:460] Webserver started at http://127.26.125.3:37209/ using document root <none> and password file <none>
I20260811 22:21:53.773437 9662 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:53.773505 9662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:53.773702 9662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:53.776028 9662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "0b6e27ae7a98492b8dd1d76402dc5ae8"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "225047b70a608fa6346457254a501b6f"
server_key_iv: "3400d65236820e449e55b2ec4481da4f"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.776604 9662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "0b6e27ae7a98492b8dd1d76402dc5ae8"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "225047b70a608fa6346457254a501b6f"
server_key_iv: "3400d65236820e449e55b2ec4481da4f"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.780838 9662 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:21:53.783640 9695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.785079 9662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:53.785208 9662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "0b6e27ae7a98492b8dd1d76402dc5ae8"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "225047b70a608fa6346457254a501b6f"
server_key_iv: "3400d65236820e449e55b2ec4481da4f"
server_key_version: "encryptionkey@0"
I20260811 22:21:53.785336 9662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:53.840139 9662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:53.843377 9662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:53.843566 9662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:53.844190 9662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:53.845093 9662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:53.845142 9662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.845197 9662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:53.845227 9662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:53.854185 9662 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:36945
I20260811 22:21:53.854264 9910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:36945 every 8 connection(s)
I20260811 22:21:53.855221 9662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:53.861644 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 9662
I20260811 22:21:53.861799 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:53.862084 27124 external_mini_cluster.cc:1428] Setting key 087a6d9d204aa58c1e4e7d0f607a3145
Aug 11 22:21:53 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486913, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:21:53.869143 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:53.857249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:46891 (local address 127.26.125.62:40321)
0811 22:21:53.857407 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:53.857412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:53.858302 (+ 890us) server_negotiation.cc:407] Connection header received
0811 22:21:53.859719 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:53.859722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:53.859780 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:53.859857 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:53.861128 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.861822 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.863250 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.863510 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.863614 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:53.866048 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:53.866051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:53.866053 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:53.866079 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:53.867306 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:53.867808 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:53.867811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:53.867812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:53.867887 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:53.868148 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:53.868150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:53.868151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:53.868313 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:53.868379 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:53.868893 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 22:21:53.869019 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:21:53.869948 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:53.857637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:46891)
0811 22:21:53.858136 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:53.858183 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:53.859494 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:53.860056 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:53.860064 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:53.860447 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:53.860980 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:53.860995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.861980 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.861984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:53.863140 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:53.863150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:53.863660 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:53.863663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:53.863829 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:53.864417 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:53.864442 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:53.865883 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:53.867412 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:53.867418 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:53.867432 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:53.867727 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:53.867948 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:53.867951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:53.867953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:53.868086 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:53.868378 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:53.868384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:53.868665 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 22:21:53.868824 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 22:21:53.869097 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":116,"threads_started":1}
I20260811 22:21:53.871073 9911 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40321
I20260811 22:21:53.871302 9911 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:53.871825 9911 heartbeater.cc:511] Master 127.26.125.62:40321 requested a full tablet report, sending...
I20260811 22:21:53.872881 9029 ts_manager.cc:194] Registered new tserver with Master: 0b6e27ae7a98492b8dd1d76402dc5ae8 (127.26.125.3:36945)
I20260811 22:21:53.873471 9029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:46891
I20260811 22:21:53.876354 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab.
Entry for principal oryx/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab.
WARNING: no policy specified for oryx/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab.
Entry for principal oryx/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab.
WARNING: no policy specified for oryx/127.26.125.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab.
Entry for principal oryx/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab.
I20260811 22:21:53.961654 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 9143
I20260811 22:21:53.976171 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 9417
I20260811 22:21:53.989418 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 9662
I20260811 22:21:54.004382 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 8967
I20260811 22:21:54.020298 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40321
--webserver_interface=127.26.125.62
--webserver_port=34737
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:54.141433 9926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:54.141726 9926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:54.141772 9926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:54.146013 9926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:54.146097 9926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:54.146116 9926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:54.146133 9926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:54.146147 9926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:54.152040 9926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=34737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:54.153275 9926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:54.154419 9926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:54.163350 9936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:54.163286 9935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:54.163373 9942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:54.163497 9926 server_base.cc:1034] running on GCE node
I20260811 22:21:54.164207 9926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:54.165405 9926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:54.166590 9926 hybrid_clock.cc:648] HybridClock initialized: now 1786486914166570 us; error 36 us; skew 500 ppm
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486914, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:54.169571 9926 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:21:54.170610 9926 webserver.cc:460] Webserver started at http://127.26.125.62:34737/ using document root <none> and password file <none>
I20260811 22:21:54.171271 9926 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:54.171368 9926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:54.175467 9926 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:21:54.177747 9956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.179078 9926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:54.179200 9926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "4bae60f74770407caf4d6baffa15719f"
format_stamp: "Formatted at 2026-08-11 22:21:52 on dist-test-slave-5bn2"
server_key: "d17d74412d56dddd9a4e55c52215051b"
server_key_iv: "3d6df757f35904c7d9cba67c6baff777"
server_key_version: "encryptionkey@0"
I20260811 22:21:54.179757 9926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:54.204631 9926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:54.208243 9926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:54.208467 9926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:54.216364 9926 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:40321
I20260811 22:21:54.216369 10035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:40321 every 8 connection(s)
I20260811 22:21:54.217628 9926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:54.218068 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 9926
I20260811 22:21:54.218559 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:35943
--local_ip_for_outbound_sockets=127.26.125.1
--tserver_master_addrs=127.26.125.62:40321
--webserver_port=35277
--webserver_interface=127.26.125.1
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.125.1
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:54.222496 10036 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:21:54.224381 10036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Bootstrap starting.
I20260811 22:21:54.241927 10036 log.cc:824] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:54.249274 10036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:21:54.249819 10036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Bootstrap complete.
I20260811 22:21:54.254333 10036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } }
I20260811 22:21:54.254812 10036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bae60f74770407caf4d6baffa15719f, State: Initialized, Role: FOLLOWER
I20260811 22:21:54.255411 10036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } }
I20260811 22:21:54.255551 10036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:54.255635 10036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:54.255736 10036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:21:54.257920 10036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } }
I20260811 22:21:54.258297 10036 leader_election.cc:302] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bae60f74770407caf4d6baffa15719f; no voters:
I20260811 22:21:54.258641 10036 leader_election.cc:288] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:21:54.258812 10047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:21:54.259121 10047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [term 2 LEADER]: Becoming Leader. State: Replica: 4bae60f74770407caf4d6baffa15719f, State: Running, Role: LEADER
I20260811 22:21:54.259485 10047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } }
I20260811 22:21:54.259961 10036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:54.260630 10047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bae60f74770407caf4d6baffa15719f. Latest consensus state: current_term: 2 leader_uuid: "4bae60f74770407caf4d6baffa15719f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } } }
I20260811 22:21:54.260766 10047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:54.261436 10048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4bae60f74770407caf4d6baffa15719f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bae60f74770407caf4d6baffa15719f" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40321 } } }
I20260811 22:21:54.261562 10048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:54.262151 10060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:54.265779 10060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:54.266564 10060 catalog_manager.cc:1259] Loaded cluster ID: 6480c044a48d459497156c13e40ea199
I20260811 22:21:54.266640 10060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:54.269305 10060 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:54.270181 10060 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4bae60f74770407caf4d6baffa15719f: Loaded TSK: 0
I20260811 22:21:54.271611 10060 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:21:54.344722 10042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:54.345005 10042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:54.345050 10042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:54.349355 10042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:54.349447 10042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:54.349514 10042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:21:54.355190 10042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.1.keytab
--principal=oryx/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:35943
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=35277
--tserver_master_addrs=127.26.125.62:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:54.356372 10042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:54.357475 10042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:54.368125 10085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:54.368127 10086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:54.368448 10092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:54.368494 10042 server_base.cc:1034] running on GCE node
I20260811 22:21:54.369131 10042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:54.369863 10042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:54.371039 10042 hybrid_clock.cc:648] HybridClock initialized: now 1786486914371002 us; error 55 us; skew 500 ppm
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486914, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:54.373894 10042 init.cc:377] Logged in from keytab as oryx/127.26.125.1@KRBTEST.COM (short username oryx)
I20260811 22:21:54.374940 10042 webserver.cc:460] Webserver started at http://127.26.125.1:35277/ using document root <none> and password file <none>
I20260811 22:21:54.375527 10042 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:54.375607 10042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:54.379647 10042 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:21:54.381851 10102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.383284 10042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:21:54.383405 10042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "328aca06c9ec46ca921273a44bdc5033"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "7a3db83b2783c59c79dbc06060df9589"
server_key_iv: "b2fe12de6f3a2dd35d6177675cbcaa3c"
server_key_version: "encryptionkey@0"
I20260811 22:21:54.384079 10042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:54.398296 10042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:54.401667 10042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:54.401857 10042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:54.402488 10042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:54.403438 10042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:54.403487 10042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.403534 10042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:54.403564 10042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.412485 10042 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:35943
I20260811 22:21:54.412511 10302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:35943 every 8 connection(s)
I20260811 22:21:54.413619 10042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:54.416903 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 10042
I20260811 22:21:54.417441 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:35759
--local_ip_for_outbound_sockets=127.26.125.2
--tserver_master_addrs=127.26.125.62:40321
--webserver_port=33341
--webserver_interface=127.26.125.2
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.125.2
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:54.427920 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.416010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:50933)
0811 22:21:54.416779 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.416840 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.417874 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.419137 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.419147 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.419579 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.420067 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.420081 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.421842 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.421845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.422836 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.422871 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.423921 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.423923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.424062 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.424635 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.424653 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.427016 (+ 2363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":153,"threads_started":1}
W20260811 22:21:54.428424 10305 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:40321 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.125.62:40321: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
W20260811 22:21:54.428808 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.415861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:50933 (local address 127.26.125.62:40321)
0811 22:21:54.416352 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.416372 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.416988 (+ 616us) server_negotiation.cc:407] Connection header received
0811 22:21:54.418306 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.418318 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.418646 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.419010 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.420253 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.421699 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.423001 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.423771 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.423964 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.427337 (+ 3373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:50933: BlockingRecv error: recv got EOF from 127.26.125.1:50933 (error 108)
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":114,"threads_started":1}
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:54.434508 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.428912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:45821)
0811 22:21:54.429371 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.429386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.429487 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.429721 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.429724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.429896 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.430019 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.430025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.430715 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.430719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.431635 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.431646 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.432037 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.432040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.432131 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.432569 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.432583 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.434378 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381}
W20260811 22:21:54.434648 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.428988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:45821 (local address 127.26.125.62:40321)
0811 22:21:54.429159 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.429163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.429453 (+ 290us) server_negotiation.cc:407] Connection header received
0811 22:21:54.429571 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.429573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.429620 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.429688 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.430118 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.430628 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.431732 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.431946 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.432023 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.434523 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:45821: BlockingRecv error: recv got EOF from 127.26.125.1:45821 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":107,"threads_started":1}
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:54.439684 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.435228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:38563)
0811 22:21:54.435373 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.435386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.435459 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.435674 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.435676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.435814 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.435880 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.435885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.436549 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.436553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.437062 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.437070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.437454 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.437457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.437527 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.437929 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.437940 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.439558 (+ 1618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 22:21:54.439774 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.435325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:38563 (local address 127.26.125.62:40321)
0811 22:21:54.435439 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.435443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.435456 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:21:54.435543 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.435545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.435587 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.435647 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.435979 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.436446 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.437163 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.437364 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.437446 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.439678 (+ 2232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:38563: BlockingRecv error: recv got EOF from 127.26.125.1:38563 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 22:21:54.440001 10305 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:21:54.538991 10309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:54.539346 10309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:54.539427 10309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:54.543766 10309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:54.543860 10309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:54.543947 10309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:21:54.549685 10309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.2.keytab
--principal=oryx/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:35759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=33341
--tserver_master_addrs=127.26.125.62:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:54.550984 10309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:54.552069 10309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:54.562582 10326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:54.562664 10309 server_base.cc:1034] running on GCE node
W20260811 22:21:54.562556 10323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:54.562563 10329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:54.563438 10309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:54.564275 10309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:54.565490 10309 hybrid_clock.cc:648] HybridClock initialized: now 1786486914565471 us; error 41 us; skew 500 ppm
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486914, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:54.568607 10309 init.cc:377] Logged in from keytab as oryx/127.26.125.2@KRBTEST.COM (short username oryx)
I20260811 22:21:54.569610 10309 webserver.cc:460] Webserver started at http://127.26.125.2:33341/ using document root <none> and password file <none>
I20260811 22:21:54.570240 10309 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:54.570322 10309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:54.574930 10309 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:21:54.577502 10343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.579133 10309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:21:54.579264 10309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "d25b213843f0463bb27b089d8a5754e9"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "21fa993156afe5d1f8816078eb8ddb99"
server_key_iv: "c476bc0e9e83149f1c1dd5582ca7486c"
server_key_version: "encryptionkey@0"
I20260811 22:21:54.579830 10309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:54.597530 10309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:54.601154 10309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:54.601325 10309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:54.601927 10309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:54.602783 10309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:54.602831 10309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.602905 10309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:54.602929 10309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.611812 10309 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:35759
I20260811 22:21:54.611852 10543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:35759 every 8 connection(s)
I20260811 22:21:54.612934 10309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:54.615865 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 10309
I20260811 22:21:54.616329 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:36945
--local_ip_for_outbound_sockets=127.26.125.3
--tserver_master_addrs=127.26.125.62:40321
--webserver_port=37209
--webserver_interface=127.26.125.3
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.125.3
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:21:54.626210 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.615029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:41515 (local address 127.26.125.62:40321)
0811 22:21:54.615176 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.615180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.616583 (+ 1403us) server_negotiation.cc:407] Connection header received
0811 22:21:54.617540 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.617544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.617611 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.617693 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.619458 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.620162 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.621545 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.621780 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.621930 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.626060 (+ 4130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:41515: BlockingRecv error: recv got EOF from 127.26.125.2:41515 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:54.626957 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.615452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:41515)
0811 22:21:54.616395 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.616461 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.617295 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.617921 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.617933 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.618484 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.619260 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.619284 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.620394 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.620397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.621398 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.621413 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.621980 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.621985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.622190 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.622958 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.622978 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.625891 (+ 2913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":126,"threads_started":1}
W20260811 22:21:54.627507 10544 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:40321 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.125.62:40321: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:54.633975 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.628033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:38097)
0811 22:21:54.628234 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.628249 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.628353 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.628641 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.628644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.628827 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.628951 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.628957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.629728 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.629731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.630650 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.630661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.631139 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.631143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.631240 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.631840 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.631857 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.633828 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121}
W20260811 22:21:54.634130 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.628138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:38097 (local address 127.26.125.62:40321)
0811 22:21:54.628252 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.628255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.628298 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:21:54.628477 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.628479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.628522 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.628595 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.629080 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.629632 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.630774 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.631040 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.631122 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.634008 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:38097: BlockingRecv error: recv got EOF from 127.26.125.2:38097 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:54.640045 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.634616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:45449)
0811 22:21:54.634742 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.634757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.634875 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.635133 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.635136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.635291 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.635368 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.635373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.636087 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.636090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.636617 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.636625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.637038 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.637042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.637115 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.637781 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.637795 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.639898 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:21:54.640146 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.634667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:45449 (local address 127.26.125.62:40321)
0811 22:21:54.634765 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.634769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.634803 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:21:54.634986 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.634989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.635040 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.635108 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.635474 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.635987 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.636723 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.636939 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.637026 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.640037 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:45449: BlockingRecv error: recv got EOF from 127.26.125.2:45449 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 22:21:54.640359 10544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:21:54.736831 10548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:54.737112 10548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:54.737156 10548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:54.741169 10548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:54.741253 10548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:54.741313 10548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:21:54.747012 10548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/krb5kdc/oryx_127.26.125.3.keytab
--principal=oryx/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:36945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=37209
--tserver_master_addrs=127.26.125.62:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:54.748201 10548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:54.749313 10548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:21:54.759744 10548 server_base.cc:1034] running on GCE node
W20260811 22:21:54.759681 10559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:54.759675 10563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:54.759680 10561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:54.760502 10548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:54.761288 10548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:54.762470 10548 hybrid_clock.cc:648] HybridClock initialized: now 1786486914762448 us; error 41 us; skew 500 ppm
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486914, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:54.766271 10548 init.cc:377] Logged in from keytab as oryx/127.26.125.3@KRBTEST.COM (short username oryx)
I20260811 22:21:54.767652 10548 webserver.cc:460] Webserver started at http://127.26.125.3:37209/ using document root <none> and password file <none>
I20260811 22:21:54.768322 10548 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:54.768411 10548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:54.772821 10548 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:21:54.775545 10582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.776979 10548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:54.777112 10548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "0b6e27ae7a98492b8dd1d76402dc5ae8"
format_stamp: "Formatted at 2026-08-11 22:21:53 on dist-test-slave-5bn2"
server_key: "225047b70a608fa6346457254a501b6f"
server_key_iv: "3400d65236820e449e55b2ec4481da4f"
server_key_version: "encryptionkey@0"
I20260811 22:21:54.777731 10548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:54.798719 10548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:54.801931 10548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:54.802126 10548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:54.802771 10548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:54.803735 10548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:54.803786 10548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.803843 10548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:54.803870 10548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:54.812867 10548 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:36945
I20260811 22:21:54.812901 10789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:36945 every 8 connection(s)
I20260811 22:21:54.813966 10548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:54.815054 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 10548
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:54.825661 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.815792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49856 (local address 127.26.125.62:40321)
0811 22:21:54.815944 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.815948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.816259 (+ 311us) server_negotiation.cc:407] Connection header received
0811 22:21:54.816430 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.816433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.816481 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.816552 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.817510 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.818281 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.819196 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.819518 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.819627 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.821377 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:54.821385 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:54.821405 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:54.821444 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:54.823355 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:54.823820 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:54.823826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:54.823832 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:54.823973 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:54.824281 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:54.824284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:54.824285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:54.824715 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:54.824942 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:54.825222 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 22:21:54.825516 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:21:54.827144 10313 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.816257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:55265 (local address 127.26.125.62:40321)
0811 22:21:54.816373 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.816386 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.817493 (+ 1107us) server_negotiation.cc:407] Connection header received
0811 22:21:54.818663 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.818666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.818720 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.818799 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.820278 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.821099 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.822899 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.823132 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.823218 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.826997 (+ 3779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:55265: BlockingRecv error: recv got EOF from 127.26.125.3:55265 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:21:54.827739 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.816569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:55265)
0811 22:21:54.817314 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.817357 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.818472 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.819011 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.819018 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.819489 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.820133 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.820155 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.821266 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.821270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.822731 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.822744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.823299 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.823303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.823521 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.824050 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.824067 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.826802 (+ 2735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":159,"threads_started":1}
W20260811 22:21:54.828256 10791 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:40321 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.125.62:40321: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:54.833590 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.828706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:55847)
0811 22:21:54.828798 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.828812 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.828889 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.829117 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.829120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.829273 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.829366 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.829372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.830076 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.830093 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.830644 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.830652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.831168 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.831171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.831262 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.831690 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.831701 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.833483 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 22:21:54.833762 10313 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.828791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:55847 (local address 127.26.125.62:40321)
0811 22:21:54.828896 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.828900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.828913 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:21:54.828967 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.828970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.829012 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.829078 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.829471 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.829976 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.830750 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.831056 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.831154 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.833636 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:55847: BlockingRecv error: recv got EOF from 127.26.125.3:55847 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:54 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:54.838694 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:54.834152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:38667)
0811 22:21:54.834271 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:54.834286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:54.834373 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:54.834582 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:54.834584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:54.834710 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:54.834785 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.834789 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.835551 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.835553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.836086 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:54.836094 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.836472 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.836474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:54.836532 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.836941 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:54.836954 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:54.838588 (+ 1634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:21:54.838892 10313 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:54.834231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:38667 (local address 127.26.125.62:40321)
0811 22:21:54.834390 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:54.834395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:54.834413 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:21:54.834452 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:54.834454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:54.834492 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:54.834556 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:54.834945 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.835454 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.836180 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:54.836388 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:54.836461 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:54.838739 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:38667: BlockingRecv error: recv got EOF from 127.26.125.3:38667 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 22:21:54.838950 10791 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 22:21:55 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:55 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:55.446450 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:55.440654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:55087)
0811 22:21:55.440955 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:55.440977 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:55.441079 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:55.441313 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:55.441315 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:55.441503 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:55.441665 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:55.441671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.442477 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.442480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:55.443141 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:55.443153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.443682 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.443686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:55.443788 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:55.444344 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:55.444358 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:55.446306 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":94,"threads_started":1}
W20260811 22:21:55.446633 10803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:55.440771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:55087 (local address 127.26.125.62:40321)
0811 22:21:55.441018 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:55.441022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:55.441037 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:55.441161 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:55.441163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:55.441209 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:55.441288 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:55.441808 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.442376 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.443278 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.443537 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.443629 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:55.446482 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:55087: BlockingRecv error: recv got EOF from 127.26.125.1:55087 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
Aug 11 22:21:55 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:55 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:55.646822 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:55.641041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:59717)
0811 22:21:55.641348 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:55.641365 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:55.641505 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:55.641812 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:55.641816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:55.642030 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:55.642138 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:55.642145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.642994 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.642997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:55.643742 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:55.643753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.644306 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.644311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:55.644396 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:55.644940 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:55.644952 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:55.646690 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":92,"threads_started":1}
W20260811 22:21:55.646998 10803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:55.641168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:59717 (local address 127.26.125.62:40321)
0811 22:21:55.641352 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:55.641357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:55.641451 (+ 94us) server_negotiation.cc:407] Connection header received
0811 22:21:55.641647 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:55.641650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:55.641702 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:55.641773 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:55.642266 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.642841 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.643860 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.644177 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.644285 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:55.646870 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:59717: BlockingRecv error: recv got EOF from 127.26.125.2:59717 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 22:21:55 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:55 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:55.845638 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:55.839726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:34189)
0811 22:21:55.840032 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:55.840048 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:55.840180 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:55.840486 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:55.840489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:55.840684 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:55.840803 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:55.840809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.841559 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.841563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:55.842135 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:55.842143 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.842590 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.842594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:55.842689 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:55.843300 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:55.843313 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:55.845435 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":93,"threads_started":1}
W20260811 22:21:55.845748 10803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:55.839833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:34189 (local address 127.26.125.62:40321)
0811 22:21:55.840028 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:55.840033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:55.840129 (+ 96us) server_negotiation.cc:407] Connection header received
0811 22:21:55.840279 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:55.840282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:55.840331 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:55.840399 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:55.840930 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.841455 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.842256 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:55.842485 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:55.842575 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:55.845643 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:34189: BlockingRecv error: recv got EOF from 127.26.125.3:34189 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 22:21:56 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:56 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:56.452782 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.447331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:44217)
0811 22:21:56.447661 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.447676 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:56.447800 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:56.448104 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:56.448109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:56.448314 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:56.448423 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.448429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.449147 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.449149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.449701 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.449708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.450104 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.450107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.450171 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.450675 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:56.450689 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:56.452646 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":130,"threads_started":1}
W20260811 22:21:56.452950 10811 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:56.447392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:44217 (local address 127.26.125.62:40321)
0811 22:21:56.447658 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:56.447663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:56.447741 (+ 78us) server_negotiation.cc:407] Connection header received
0811 22:21:56.447909 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:56.447913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:56.447973 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:56.448068 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:56.448530 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.449062 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.449786 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.450009 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.450093 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.452806 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:44217: BlockingRecv error: recv got EOF from 127.26.125.1:44217 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
Aug 11 22:21:56 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:56 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:56.657966 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.647737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:41637)
0811 22:21:56.648058 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.648073 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:56.648194 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:56.648484 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:56.648487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:56.648670 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:56.648768 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.648773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.649634 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.649637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.650241 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.650251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.650714 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.650717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.650812 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.655751 (+ 4939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:56.655767 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:56.657816 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":123,"threads_started":1}
W20260811 22:21:56.658191 10811 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:56.647794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:41637 (local address 127.26.125.62:40321)
0811 22:21:56.647970 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:56.647974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:56.648148 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:21:56.648286 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:56.648289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:56.648352 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:56.648446 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:56.648917 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.649516 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.650377 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.650614 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.650703 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.658051 (+ 7348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:41637: BlockingRecv error: recv got EOF from 127.26.125.2:41637 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 22:21:56 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:56 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:56.853111 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.846663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:39355)
0811 22:21:56.847058 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.847077 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:56.847199 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:56.847489 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:56.847493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:56.847693 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:56.847802 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.847809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.848626 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.848629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.849223 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.849233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.849747 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.849751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.849831 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.850408 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:56.850421 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:56.852940 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":115,"threads_started":1}
W20260811 22:21:56.853286 10811 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:56.846843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:39355 (local address 127.26.125.62:40321)
0811 22:21:56.847047 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:56.847050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:56.847154 (+ 104us) server_negotiation.cc:407] Connection header received
0811 22:21:56.847288 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:56.847291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:56.847350 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:56.847432 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:56.847952 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.848496 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.849351 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.849609 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.849715 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.853146 (+ 3431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:39355: BlockingRecv error: recv got EOF from 127.26.125.3:39355 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 22:21:57 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:57 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:57.459707 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.453585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:57117)
0811 22:21:57.453896 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.453912 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:57.454008 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:57.454356 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:57.454359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:57.454558 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:57.454677 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.454683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.455560 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.455564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.456174 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.456184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.456751 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.456755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.456828 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.457352 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:57.457365 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:57.459558 (+ 2193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":98,"threads_started":1}
W20260811 22:21:57.460052 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:57.453821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:57117 (local address 127.26.125.62:40321)
0811 22:21:57.454087 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.454091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.454105 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:57.454145 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.454148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.454193 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.454285 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.454809 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.455425 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.456318 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.456620 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.456716 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.459776 (+ 3060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:57117: BlockingRecv error: recv got EOF from 127.26.125.1:57117 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
Aug 11 22:21:57 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:57 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:57.664649 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.658781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:40733)
0811 22:21:57.659118 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.659135 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:57.659252 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:57.659520 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:57.659523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:57.659733 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:57.659830 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.659836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.660653 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.660657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.661340 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.661373 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.661811 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.661815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.661895 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.662510 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:57.662523 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:57.664495 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":116,"threads_started":1}
W20260811 22:21:57.664769 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:57.658957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:40733 (local address 127.26.125.62:40321)
0811 22:21:57.659123 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.659128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.659192 (+ 64us) server_negotiation.cc:407] Connection header received
0811 22:21:57.659347 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.659350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.659405 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.659476 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.659955 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.660548 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.661481 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.661710 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.661798 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.664659 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:40733: BlockingRecv error: recv got EOF from 127.26.125.2:40733 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:21:57 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:57 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:57.859871 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.854063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:59153)
0811 22:21:57.854438 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.854455 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:57.854607 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:57.854926 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:57.854929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:57.855150 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:57.855249 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.855255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.856041 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.856044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.856643 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.856652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.857111 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.857114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.857193 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.857725 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:57.857737 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:57.859679 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":93,"threads_started":1}
W20260811 22:21:57.859968 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:57.854227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:59153 (local address 127.26.125.62:40321)
0811 22:21:57.854409 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.854419 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.854543 (+ 124us) server_negotiation.cc:407] Connection header received
0811 22:21:57.854711 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.854714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.854767 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.854881 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.855382 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.855926 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.856766 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.857001 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.857095 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.859861 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:59153: BlockingRecv error: recv got EOF from 127.26.125.3:59153 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:21:58 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:58 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:58.466439 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:58.460695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:34609)
0811 22:21:58.460968 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:58.460982 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:58.461124 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:58.461420 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:58.461422 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:58.461613 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:58.461739 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:58.461745 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.462580 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.462584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:58.463293 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:58.463304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.463811 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.463815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:58.463919 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:58.464485 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:58.464500 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:58.466287 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
W20260811 22:21:58.466576 10824 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:58.460748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:34609 (local address 127.26.125.62:40321)
0811 22:21:58.461017 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:58.461021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:58.461043 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:21:58.461242 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:58.461245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:58.461301 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:58.461391 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:58.461847 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.462480 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.463418 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.463708 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.463802 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:58.466454 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:34609: BlockingRecv error: recv got EOF from 127.26.125.1:34609 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
Aug 11 22:21:58 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:58 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:58.671218 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:58.665395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:58205)
0811 22:21:58.665787 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:58.665799 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:58.665915 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:58.666203 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:58.666207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:58.666428 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:58.666540 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:58.666546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.667367 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.667370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:58.667993 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:58.668004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.668443 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.668447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:58.668559 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:58.669185 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:58.669197 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:58.671089 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":138,"threads_started":1}
W20260811 22:21:58.671434 10824 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:58.665452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:58205 (local address 127.26.125.62:40321)
0811 22:21:58.665604 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:58.665609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:58.665871 (+ 262us) server_negotiation.cc:407] Connection header received
0811 22:21:58.666034 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:58.666036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:58.666088 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:58.666152 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:58.666659 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.667246 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.668110 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.668336 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.668427 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:58.671297 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:58205: BlockingRecv error: recv got EOF from 127.26.125.2:58205 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 22:21:58 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:58 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:58.866533 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:58.860890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:39035)
0811 22:21:58.861198 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:58.861213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:58.861341 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:58.861602 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:58.861605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:58.861814 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:58.861911 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:58.861918 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.862713 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.862716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:58.863356 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:58.863366 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.863813 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.863817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:58.863897 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:58.864396 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:58.864407 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:58.866374 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
W20260811 22:21:58.866688 10824 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:58.860957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:39035 (local address 127.26.125.62:40321)
0811 22:21:58.861111 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:58.861116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:58.861279 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:21:58.861430 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:58.861433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:58.861484 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:58.861556 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:58.862058 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.862596 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.863474 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:58.863707 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:58.863799 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:58.866567 (+ 2768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:39035: BlockingRecv error: recv got EOF from 127.26.125.3:39035 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 22:21:59 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:59 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:59.473372 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:59.467334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:49107)
0811 22:21:59.467621 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:59.467636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:59.467731 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:59.467968 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:59.467971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:59.468159 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:59.468256 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:59.468261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.469099 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.469103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:59.470019 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:59.470033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.470537 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.470540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:59.470614 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:59.471198 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:59.471211 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:59.473244 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
W20260811 22:21:59.473557 10834 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:59.467436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:49107 (local address 127.26.125.62:40321)
0811 22:21:59.467740 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:59.467744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:59.467780 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:21:59.467818 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:59.467820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:59.467863 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:59.467928 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:59.468398 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.468983 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.470155 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.470391 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.470478 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:59.473418 (+ 2940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:49107: BlockingRecv error: recv got EOF from 127.26.125.1:49107 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":152,"threads_started":1}
Aug 11 22:21:59 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:59 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:59.677608 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:59.672115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:49123)
0811 22:21:59.672440 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:59.672454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:59.672568 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:59.672783 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:59.672785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:59.672977 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:59.673119 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:59.673126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.673809 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.673812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:59.674409 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:59.674417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.674880 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.674884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:59.674955 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:59.675454 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:59.675466 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:59.677461 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":92,"threads_started":1}
W20260811 22:21:59.677772 10834 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:59.672331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:49123 (local address 127.26.125.62:40321)
0811 22:21:59.672479 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:59.672484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:59.672518 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:21:59.672650 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:59.672653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:59.672694 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:59.672751 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:59.673216 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.673729 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.674522 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.674759 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.674836 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:59.677646 (+ 2810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:49123: BlockingRecv error: recv got EOF from 127.26.125.2:49123 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 22:21:59 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:21:59 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:21:59.873523 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:59.867699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:41957)
0811 22:21:59.867992 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:59.868006 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:59.868114 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:59.868514 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:59.868517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:59.868759 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:59.868883 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:59.868890 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.869753 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.869758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:59.870470 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:59.870482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.870975 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.870979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:59.871049 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:59.871558 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:59.871570 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:59.873378 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
W20260811 22:21:59.873742 10834 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:59.867825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:41957 (local address 127.26.125.62:40321)
0811 22:21:59.867968 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:59.867971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:59.868078 (+ 107us) server_negotiation.cc:407] Connection header received
0811 22:21:59.868311 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:59.868314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:59.868372 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:59.868458 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:59.869010 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.869639 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.870591 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:59.870813 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:59.870942 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:59.873606 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:41957: BlockingRecv error: recv got EOF from 127.26.125.3:41957 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 22:22:00 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:00 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:00.480062 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:00.474184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:56087)
0811 22:22:00.474573 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:00.474590 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:00.474696 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:00.475026 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:00.475029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:00.475263 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:00.475379 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:00.475386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.476188 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.476192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:00.476748 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:00.476756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.477203 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.477207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:00.477285 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:00.477774 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:00.477786 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:00.479888 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":89,"threads_started":1}
W20260811 22:22:00.480181 10839 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:00.474327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:56087 (local address 127.26.125.62:40321)
0811 22:22:00.474667 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:00.474672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:00.474688 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:22:00.474800 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:00.474803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:00.474875 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:00.474957 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:00.475497 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.476075 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.476863 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.477097 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.477185 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:00.480046 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:56087: BlockingRecv error: recv got EOF from 127.26.125.1:56087 (error 108)
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":81,"threads_started":1}
Aug 11 22:22:00 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:00 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:00.684796 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:00.678627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:49071)
0811 22:22:00.678991 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:00.679007 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:00.679129 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:00.679457 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:00.679460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:00.679689 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:00.679795 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:00.679801 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.680735 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.680739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:00.681413 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:00.681426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.681895 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.681898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:00.681968 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:00.682561 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:00.682573 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:00.684667 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":94,"threads_started":1}
W20260811 22:22:00.685060 10839 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:00.678787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:49071 (local address 127.26.125.62:40321)
0811 22:22:00.678992 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:00.678996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:00.679092 (+ 96us) server_negotiation.cc:407] Connection header received
0811 22:22:00.679215 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:00.679218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:00.679266 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:00.679419 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:00.679908 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.680592 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.681530 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.681767 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.681896 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:00.684920 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:49071: BlockingRecv error: recv got EOF from 127.26.125.2:49071 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 22:22:00 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:00 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:00.880304 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:00.874335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:33805)
0811 22:22:00.874663 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:00.874678 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:00.874785 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:00.875123 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:00.875126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:00.875321 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:00.875435 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:00.875442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.876335 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.876338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:00.876943 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:00.876953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.877533 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.877537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:00.877619 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:00.878142 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:00.878154 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:00.880141 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":137,"threads_started":1}
W20260811 22:22:00.880410 10839 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:00.874441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:33805 (local address 127.26.125.62:40321)
0811 22:22:00.874616 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:00.874620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:00.874743 (+ 123us) server_negotiation.cc:407] Connection header received
0811 22:22:00.874890 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:00.874892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:00.874950 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:00.875026 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:00.875622 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.876199 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.877079 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:00.877316 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:00.877472 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:00.880302 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:33805: BlockingRecv error: recv got EOF from 127.26.125.3:33805 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 22:22:01 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:01 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:01.486814 10844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:01.480953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:60433)
0811 22:22:01.481347 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:01.481361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:01.481478 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:01.481764 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:01.481767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:01.481958 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:01.482075 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:01.482081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.482881 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.482886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:01.483469 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:01.483479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.484081 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.484084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:01.484153 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:01.484653 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:01.484665 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:01.486686 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":86,"threads_started":1}
W20260811 22:22:01.487094 10845 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:01.481067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:60433 (local address 127.26.125.62:40321)
0811 22:22:01.481340 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:01.481344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:01.481433 (+ 89us) server_negotiation.cc:407] Connection header received
0811 22:22:01.481557 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:01.481560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:01.481607 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:01.481691 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:01.482174 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.482747 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.483602 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.483958 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.484049 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:01.486935 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:60433: BlockingRecv error: recv got EOF from 127.26.125.1:60433 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":70,"threads_started":1}
Aug 11 22:22:01 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:01 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:01.696465 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:01.685581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:51999)
0811 22:22:01.685897 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:01.685911 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:01.686020 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:01.686246 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:01.686248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:01.686430 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:01.686560 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:01.686566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.687368 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.687371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:01.687957 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:01.687966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.688430 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.688433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:01.688499 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:01.694048 (+ 5549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:01.694064 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:01.696341 (+ 2277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":73,"threads_started":1}
W20260811 22:22:01.696713 10845 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:01.685687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:51999 (local address 127.26.125.62:40321)
0811 22:22:01.685832 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:01.685836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:01.686000 (+ 164us) server_negotiation.cc:407] Connection header received
0811 22:22:01.686098 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:01.686100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:01.686151 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:01.686218 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:01.686655 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.687272 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.688096 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.688325 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.688421 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:01.696598 (+ 8177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:51999: BlockingRecv error: recv got EOF from 127.26.125.2:51999 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 22:22:01 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:01 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:01.888022 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:01.881228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:52213)
0811 22:22:01.881614 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:01.881629 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:01.881759 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:01.882076 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:01.882079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:01.882275 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:01.882374 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:01.882380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.883551 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.883555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:01.884553 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:01.884569 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.885101 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.885104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:01.885172 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:01.885678 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:01.885695 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:01.887878 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":105,"threads_started":1}
W20260811 22:22:01.888204 10845 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:01.881371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:52213 (local address 127.26.125.62:40321)
0811 22:22:01.881524 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:01.881528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:01.881714 (+ 186us) server_negotiation.cc:407] Connection header received
0811 22:22:01.881860 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:01.881864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:01.881927 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:01.882015 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:01.882572 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.883426 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.884696 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.884997 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.885089 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:01.888065 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:52213: BlockingRecv error: recv got EOF from 127.26.125.3:52213 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 22:22:02 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:02 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:02.493486 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:02.487706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:58401)
0811 22:22:02.487996 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:02.488012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:02.488113 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:02.488343 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:02.488345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:02.488536 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:02.488637 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:02.488642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.489441 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.489443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:02.490029 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:02.490038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.490575 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.490578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:02.490640 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:02.491166 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:02.491178 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:02.493324 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
W20260811 22:22:02.493672 10851 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:02.487870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:58401 (local address 127.26.125.62:40321)
0811 22:22:02.488128 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:02.488132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:02.488147 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:22:02.488190 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:02.488192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:02.488236 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:02.488309 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:02.488732 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.489351 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.490179 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.490470 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.490571 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:02.493529 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:58401: BlockingRecv error: recv got EOF from 127.26.125.1:58401 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":92,"threads_started":1}
Aug 11 22:22:02 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:02 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:02.703027 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:02.697312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:44695)
0811 22:22:02.697621 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:02.697634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:02.697745 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:02.698034 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:02.698038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:02.698268 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:02.698406 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:02.698416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.699363 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.699367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:02.699955 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:02.699964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.700410 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.700415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:02.700487 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:02.700977 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:02.700988 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:02.702886 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":124,"threads_started":1}
W20260811 22:22:02.703186 10851 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:02.697363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:44695 (local address 127.26.125.62:40321)
0811 22:22:02.697513 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:02.697518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:02.697708 (+ 190us) server_negotiation.cc:407] Connection header received
0811 22:22:02.697836 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:02.697839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:02.697895 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:02.697976 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:02.698538 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.699256 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.700058 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.700272 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.700345 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:02.703060 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:44695: BlockingRecv error: recv got EOF from 127.26.125.2:44695 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 22:22:02 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:02 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:02.894835 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:02.888890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:44081)
0811 22:22:02.889209 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:02.889224 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:02.889340 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:02.889645 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:02.889649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:02.889844 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:02.889984 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:02.889992 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.890915 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.890918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:02.891506 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:02.891517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.891941 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.891944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:02.892007 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:02.892649 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:02.892661 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:02.894690 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":114,"threads_started":1}
W20260811 22:22:02.895001 10851 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:02.888946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:44081 (local address 127.26.125.62:40321)
0811 22:22:02.889121 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:02.889126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:02.889295 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:22:02.889442 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:02.889444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:02.889495 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:02.889590 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:02.890143 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.890747 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.891622 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:02.891847 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:02.891929 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:02.894866 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:44081: BlockingRecv error: recv got EOF from 127.26.125.3:44081 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 22:22:03 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:03 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:03.499964 10856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:03.494316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:42661)
0811 22:22:03.494610 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:03.494627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:03.494731 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:03.494998 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:03.495000 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:03.495181 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:03.495281 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:03.495286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.496019 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.496022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:03.496587 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:03.496596 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.497075 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.497078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:03.497136 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:03.497676 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:03.497689 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:03.499812 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
W20260811 22:22:03.500119 10857 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:03.494458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:42661 (local address 127.26.125.62:40321)
0811 22:22:03.494714 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:03.494717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:03.494732 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:22:03.494804 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:03.494806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:03.494892 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:03.494972 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:03.495370 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.495929 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.496717 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.496975 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.497063 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:03.499979 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:42661: BlockingRecv error: recv got EOF from 127.26.125.1:42661 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":84,"threads_started":1}
Aug 11 22:22:03 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:03 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:03.709906 10860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:03.704001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:53797)
0811 22:22:03.704365 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:03.704380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:03.704513 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:03.704806 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:03.704809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:03.705002 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:03.705103 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:03.705109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.705919 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.705923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:03.706497 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:03.706506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.707074 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.707078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:03.707177 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:03.707783 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:03.707796 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:03.709713 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":89,"threads_started":1}
W20260811 22:22:03.709987 10857 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:03.704143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:53797 (local address 127.26.125.62:40321)
0811 22:22:03.704323 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:03.704328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:03.704504 (+ 176us) server_negotiation.cc:407] Connection header received
0811 22:22:03.704590 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:03.704593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:03.704646 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:03.704727 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:03.705203 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.705790 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.706584 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.706930 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.707032 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:03.709879 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:53797: BlockingRecv error: recv got EOF from 127.26.125.2:53797 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 22:22:03 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:03 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:03.901222 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:03.895800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:45829)
0811 22:22:03.896145 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:03.896158 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:03.896264 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:03.896506 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:03.896508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:03.896704 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:03.896807 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:03.896813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.897619 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.897622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:03.898202 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:03.898212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.898639 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.898642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:03.898706 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:03.899266 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:03.899279 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:03.901094 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":100,"threads_started":1}
W20260811 22:22:03.901364 10857 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:03.895886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:45829 (local address 127.26.125.62:40321)
0811 22:22:03.896042 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:03.896046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:03.896221 (+ 175us) server_negotiation.cc:407] Connection header received
0811 22:22:03.896357 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:03.896360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:03.896409 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:03.896483 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:03.896934 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.897485 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.898308 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:03.898538 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:03.898616 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:03.901246 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:45829: BlockingRecv error: recv got EOF from 127.26.125.3:45829 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 22:22:04 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:04 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:04.507290 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:04.501057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:55623)
0811 22:22:04.501428 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:04.501446 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:04.501615 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:04.501971 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:04.501975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:04.502205 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:04.502338 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:04.502346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.503413 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.503415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:04.503987 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:04.503997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.504560 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.504563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:04.504635 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:04.505208 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:04.505220 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:04.507158 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":109,"threads_started":1}
W20260811 22:22:04.507499 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:04.501221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:55623 (local address 127.26.125.62:40321)
0811 22:22:04.501499 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:04.501503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:04.501522 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:22:04.501748 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:04.501752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:04.501818 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:04.501914 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:04.502496 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.503299 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.504136 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.504434 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.504523 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:04.507342 (+ 2819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:55623: BlockingRecv error: recv got EOF from 127.26.125.1:55623 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":88,"threads_started":1}
Aug 11 22:22:04 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:04 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:04.716405 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:04.710930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:47839)
0811 22:22:04.711238 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:04.711255 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:04.711378 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:04.711674 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:04.711678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:04.711889 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:04.712008 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:04.712015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.712776 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.712780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:04.713348 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:04.713356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.713807 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.713811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:04.713892 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:04.714419 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:04.714431 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:04.716237 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":111,"threads_started":1}
W20260811 22:22:04.716512 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:04.711081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:47839 (local address 127.26.125.62:40321)
0811 22:22:04.711241 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:04.711246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:04.711331 (+ 85us) server_negotiation.cc:407] Connection header received
0811 22:22:04.711508 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:04.711510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:04.711555 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:04.711630 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:04.712137 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.712669 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.713469 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.713697 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.713792 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:04.716394 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:47839: BlockingRecv error: recv got EOF from 127.26.125.2:47839 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 22:22:04 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:04 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:04.908108 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:04.902035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:38941)
0811 22:22:04.902357 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:04.902376 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:04.902506 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:04.902764 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:04.902768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:04.903012 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:04.903117 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:04.903123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.903981 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.903985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:04.904547 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:04.904555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.905023 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.905028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:04.905110 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:04.905664 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:04.905678 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:04.907894 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":84,"threads_started":1}
W20260811 22:22:04.908215 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:04.902227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:38941 (local address 127.26.125.62:40321)
0811 22:22:04.902378 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:04.902383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:04.902458 (+ 75us) server_negotiation.cc:407] Connection header received
0811 22:22:04.902598 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:04.902601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:04.902649 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:04.902717 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:04.903241 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.903830 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.904671 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.904909 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.905003 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:04.908095 (+ 3092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:38941: BlockingRecv error: recv got EOF from 127.26.125.3:38941 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 22:22:05 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:05 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:05.514722 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:05.508274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:37019)
0811 22:22:05.508639 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:05.508656 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:05.508777 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:05.509144 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:05.509147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:05.509340 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:05.509447 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.509452 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.510329 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.510332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.511055 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.511067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.511636 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.511640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.511713 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.512394 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:05.512472 (+ 78us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:05.514584 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":96,"threads_started":1}
W20260811 22:22:05.514927 10872 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:05.508422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:37019 (local address 127.26.125.62:40321)
0811 22:22:05.508669 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:05.508673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:05.508749 (+ 76us) server_negotiation.cc:407] Connection header received
0811 22:22:05.508917 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:05.508920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:05.508977 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:05.509072 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:05.509643 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.510212 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.511185 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.511500 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.511599 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.514755 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:37019: BlockingRecv error: recv got EOF from 127.26.125.1:37019 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":65,"threads_started":1}
Aug 11 22:22:05 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:05 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:05.723076 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:05.717302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:50337)
0811 22:22:05.717596 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:05.717613 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:05.717753 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:05.718031 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:05.718034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:05.718261 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:05.718375 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.718385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.719238 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.719242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.719821 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.719841 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.720305 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.720309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.720398 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.720991 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:05.721009 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:05.722929 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":91,"threads_started":1}
W20260811 22:22:05.723443 10872 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:05.717412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:50337 (local address 127.26.125.62:40321)
0811 22:22:05.717571 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:05.717576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:05.717680 (+ 104us) server_negotiation.cc:407] Connection header received
0811 22:22:05.717864 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:05.717868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:05.717924 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:05.718008 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:05.718511 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.719128 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.719956 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.720185 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.720277 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.723113 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:50337: BlockingRecv error: recv got EOF from 127.26.125.2:50337 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 22:22:05 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:05 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:05.914476 10876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:05.908973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:55467)
0811 22:22:05.909260 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:05.909274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:05.909394 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:05.909719 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:05.909721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:05.909911 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:05.910014 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.910020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.910839 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.910842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.911435 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.911445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.911871 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.911873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.911930 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.912426 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:05.912437 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:05.914346 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":85,"threads_started":1}
W20260811 22:22:05.914642 10872 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:05.909055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:55467 (local address 127.26.125.62:40321)
0811 22:22:05.909194 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:05.909199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:05.909347 (+ 148us) server_negotiation.cc:407] Connection header received
0811 22:22:05.909524 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:05.909534 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:05.909589 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:05.909673 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:05.910140 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.910738 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.911551 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.911780 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.911866 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.914514 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:55467: BlockingRecv error: recv got EOF from 127.26.125.3:55467 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 22:22:06 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:06 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:06.521369 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.515787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:47129)
0811 22:22:06.516080 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.516095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.516200 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.516513 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.516516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.516708 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:06.516813 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.516818 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.517606 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.517610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.518182 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.518191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.518615 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.518618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.518685 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.519234 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:06.519247 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:06.521248 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":97,"threads_started":1}
W20260811 22:22:06.521564 10878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:06.515944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:47129 (local address 127.26.125.62:40321)
0811 22:22:06.516236 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.516240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.516259 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:22:06.516302 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.516304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.516347 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.516427 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:06.516926 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.517472 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.518275 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.518493 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.518587 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.521427 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:47129: BlockingRecv error: recv got EOF from 127.26.125.1:47129 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":122,"threads_started":1}
Aug 11 22:22:06 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:06 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:06.732942 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.724022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:40385)
0811 22:22:06.724296 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.724310 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.724420 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.724701 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.724704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.724937 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:06.725049 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.725056 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.726037 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.726042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.726661 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.726672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.727177 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.727180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.727249 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.731051 (+ 3802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:06.731064 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:06.732816 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":77,"threads_started":1}
W20260811 22:22:06.733083 10878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:06.724157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:40385 (local address 127.26.125.62:40321)
0811 22:22:06.724300 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.724304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.724370 (+ 66us) server_negotiation.cc:407] Connection header received
0811 22:22:06.724514 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.724517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.724573 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.724676 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:06.725159 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.725889 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.726781 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.727064 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.727144 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.732959 (+ 5815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:40385: BlockingRecv error: recv got EOF from 127.26.125.2:40385 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 22:22:06 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:06 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:06.921429 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.915526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:34753)
0811 22:22:06.915836 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.915853 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.915989 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.916290 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.916294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.916548 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:06.916651 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.916658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.917431 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.917434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.918038 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.918050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.918664 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.918668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.918759 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.919379 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:06.919394 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:06.921303 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
W20260811 22:22:06.921700 10878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:06.915676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:34753 (local address 127.26.125.62:40321)
0811 22:22:06.915839 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.915844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.915934 (+ 90us) server_negotiation.cc:407] Connection header received
0811 22:22:06.916108 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.916111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.916162 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.916254 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:06.916781 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.917319 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.918184 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.918532 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.918646 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.921560 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:34753: BlockingRecv error: recv got EOF from 127.26.125.3:34753 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 22:22:07 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:07 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:07.528188 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:07.522282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:50745)
0811 22:22:07.522570 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:07.522584 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:07.522686 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:07.522969 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:07.522971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:07.523162 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:07.523264 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:07.523269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.524030 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.524034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:07.524606 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:07.524616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.525135 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.525138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:07.525208 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:07.525765 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:07.525777 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:07.528048 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
W20260811 22:22:07.528366 10886 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:07.522368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:50745 (local address 127.26.125.62:40321)
0811 22:22:07.522646 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:07.522650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:07.522665 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:22:07.522783 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:07.522785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:07.522833 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:07.522929 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:07.523368 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.523900 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.524746 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.524982 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.525095 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:07.528221 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:50745: BlockingRecv error: recv got EOF from 127.26.125.1:50745 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
Aug 11 22:22:07 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:07 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:07.739382 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:07.733767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:41005)
0811 22:22:07.734077 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:07.734092 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:07.734191 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:07.734392 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:07.734394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:07.734575 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:07.734675 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:07.734680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.735470 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.735473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:07.736072 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:07.736082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.736558 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.736561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:07.736653 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:07.737281 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:07.737295 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:07.739253 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":115,"threads_started":1}
W20260811 22:22:07.739558 10886 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:07.733912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:41005 (local address 127.26.125.62:40321)
0811 22:22:07.734078 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:07.734081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:07.734152 (+ 71us) server_negotiation.cc:407] Connection header received
0811 22:22:07.734259 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:07.734261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:07.734307 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:07.734377 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:07.734762 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.735343 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.736190 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.736419 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.736502 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:07.739431 (+ 2929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:41005: BlockingRecv error: recv got EOF from 127.26.125.2:41005 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 22:22:07 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:07 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:07.928305 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:07.922311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:33569)
0811 22:22:07.922620 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:07.922635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:07.922743 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:07.923026 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:07.923029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:07.923220 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:07.923319 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:07.923324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.924134 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.924137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:07.924746 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:07.924765 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.925270 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.925274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:07.925348 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:07.925914 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:07.925931 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:07.928118 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":100,"threads_started":1}
W20260811 22:22:07.928402 10886 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:07.922426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:33569 (local address 127.26.125.62:40321)
0811 22:22:07.922595 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:07.922599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:07.922704 (+ 105us) server_negotiation.cc:407] Connection header received
0811 22:22:07.922822 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:07.922824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:07.922909 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:07.923000 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:07.923483 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.924019 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.924879 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:07.925133 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:07.925213 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:07.928281 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:33569: BlockingRecv error: recv got EOF from 127.26.125.3:33569 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 22:22:08 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:08 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:08.535652 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:08.529182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:40851)
0811 22:22:08.529630 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:08.529651 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:08.529772 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:08.530061 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:08.530063 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:08.530274 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:08.530380 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.530386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.531278 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.531281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.531872 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.531882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.532470 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.532474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.532601 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.533308 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:08.533325 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:08.535431 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":83,"threads_started":1}
W20260811 22:22:08.535732 10891 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:08.529329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:40851 (local address 127.26.125.62:40321)
0811 22:22:08.529713 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:08.529717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:08.529733 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:22:08.529861 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:08.529864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:08.529910 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:08.530024 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:08.530509 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.531158 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.532016 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.532330 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.532465 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.535608 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:40851: BlockingRecv error: recv got EOF from 127.26.125.1:40851 (error 108)
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":77,"threads_started":1}
Aug 11 22:22:08 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:08 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:08.746150 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:08.740302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:42809)
0811 22:22:08.740649 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:08.740666 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:08.740793 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:08.741067 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:08.741069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:08.741268 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:08.741398 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.741406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.742242 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.742246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.742888 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.742901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.743391 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.743398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.743464 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.743961 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:08.743972 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:08.745968 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":95,"threads_started":1}
W20260811 22:22:08.746263 10891 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:08.740483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:42809 (local address 127.26.125.62:40321)
0811 22:22:08.740654 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:08.740658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:08.740732 (+ 74us) server_negotiation.cc:407] Connection header received
0811 22:22:08.740904 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:08.740907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:08.740954 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:08.741035 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:08.741560 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.742110 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.743031 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.743265 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.743373 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.746151 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:42809: BlockingRecv error: recv got EOF from 127.26.125.2:42809 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 22:22:08 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:08 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.3@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:08.935230 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:08.929248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.3:52075)
0811 22:22:08.929557 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:08.929572 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:08.929686 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:08.929978 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:08.929981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:08.930198 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:08.930306 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.930311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.931226 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.931229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.931820 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.931829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.932327 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.932330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.932418 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.933021 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:08.933038 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:08.935102 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":84,"threads_started":1}
W20260811 22:22:08.935420 10891 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:08.929420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:52075 (local address 127.26.125.62:40321)
0811 22:22:08.929557 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:08.929561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:08.929674 (+ 113us) server_negotiation.cc:407] Connection header received
0811 22:22:08.929759 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:08.929762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:08.929815 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:08.929896 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:08.930470 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.931101 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.931942 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.932201 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.932283 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.935285 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.3:52075: BlockingRecv error: recv got EOF from 127.26.125.3:52075 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 22:22:09 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:09 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.1@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:09.542986 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.536728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.1:35499)
0811 22:22:09.537042 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.537060 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:09.537168 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:09.537438 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:09.537441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:09.537690 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:09.537798 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.537804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.538641 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.538644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.539272 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.539282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.539802 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.539805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.539874 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.540449 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:09.540510 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:09.542769 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":94,"threads_started":1}
W20260811 22:22:09.543313 10895 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:09.536876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:35499 (local address 127.26.125.62:40321)
0811 22:22:09.537178 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.537182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.537220 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:22:09.537260 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.537263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.537313 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.537381 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:09.537939 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.538496 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.539406 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.539674 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.539767 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.543032 (+ 3265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.1:35499: BlockingRecv error: recv got EOF from 127.26.125.1:35499 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":94,"threads_started":1}
Aug 11 22:22:09 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:22:09 dist-test-slave-5bn2 krb5kdc[8943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.125.2@KRBTEST.COM for oryx/127.26.125.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:22:09.753276 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.747284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40321 (local address 127.26.125.2:39045)
0811 22:22:09.747640 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.747656 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:09.747770 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:09.748150 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:09.748153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:09.748342 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:09.748452 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.748459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.749234 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.749237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.749844 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.749853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.750284 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.750286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.750356 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.750932 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:09.750947 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:09.753105 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.125.62:40321: Server oryx/127.26.125.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":115,"threads_started":1}
W20260811 22:22:09.753356 10895 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:09.747539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:39045 (local address 127.26.125.62:40321)
0811 22:22:09.747688 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.747692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.747713 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:22:09.747908 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.747911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.747967 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.748054 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:09.748574 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.749123 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.749944 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.750191 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.750270 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.753244 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.125.2:39045: BlockingRecv error: recv got EOF from 127.26.125.2:39045 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:22:09.823393 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 10042
I20260811 22:22:09.835340 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 10309
I20260811 22:22:09.872627 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 10548
I20260811 22:22:09.892678 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 9926
2026-08-11T22:22:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18400 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T22:22:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:22:09Z Disabled control of system clock
I20260811 22:22:09.932363 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:46083
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:41759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:46083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:22:10.051929 10903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:10.052237 10903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:10.052273 10903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:10.056427 10903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:10.056511 10903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:10.056527 10903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:10.056543 10903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:10.056559 10903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:10.061638 10903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:46083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:46083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:10.062943 10903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:10.064502 10903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:10.073990 10913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:10.073958 10923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:10.074136 10903 server_base.cc:1034] running on GCE node
W20260811 22:22:10.073963 10912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:10.075141 10903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:10.076367 10903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:10.077562 10903 hybrid_clock.cc:648] HybridClock initialized: now 1786486930077530 us; error 49 us; skew 500 ppm
I20260811 22:22:10.079730 10903 webserver.cc:460] Webserver started at http://127.26.125.62:41271/ using document root <none> and password file <none>
I20260811 22:22:10.080356 10903 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:10.080439 10903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:10.080675 10903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:10.082988 10903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "18619927769b4f1aa4d1323ee9d66424"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "88ec307a91fce26fd7427fc7c6d2701a"
server_key_iv: "f575f3aaa07953861d3f179d93f59ae3"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.083624 10903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "18619927769b4f1aa4d1323ee9d66424"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "88ec307a91fce26fd7427fc7c6d2701a"
server_key_iv: "f575f3aaa07953861d3f179d93f59ae3"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.087623 10903 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:22:10.090207 10936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.091607 10903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:22:10.091745 10903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "18619927769b4f1aa4d1323ee9d66424"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "88ec307a91fce26fd7427fc7c6d2701a"
server_key_iv: "f575f3aaa07953861d3f179d93f59ae3"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.091854 10903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:10.111977 10903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:10.112831 10903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:10.113059 10903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:10.120604 10903 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:46083
I20260811 22:22:10.120657 11029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:46083 every 8 connection(s)
I20260811 22:22:10.121811 10903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:10.125424 11031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:10.129427 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 10903
I20260811 22:22:10.129578 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:22:10.129901 27124 external_mini_cluster.cc:1428] Setting key a2c61a50bbd6c845fd6855edecf85a30
I20260811 22:22:10.132426 11031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424: Bootstrap starting.
I20260811 22:22:10.134936 11031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:10.135735 11031 log.cc:824] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:10.137856 11031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424: No bootstrap required, opened a new log
I20260811 22:22:10.140591 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.131683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.26.125.62:46083)
0811 22:22:10.132449 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:10.132467 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:10.132508 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:22:10.133214 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:10.133234 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:10.133700 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:10.134076 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:10.134550 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.135632 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.136581 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.137081 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.137232 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:10.138150 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:10.138157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:10.138175 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:22:10.138696 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:10.138780 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:22:10.138810 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:10.138933 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:10.139122 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 22:22:10.139546 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":163,"threads_started":1}
I20260811 22:22:10.142380 11031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18619927769b4f1aa4d1323ee9d66424" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46083 } }
I20260811 22:22:10.142697 11031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:10.142776 11031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18619927769b4f1aa4d1323ee9d66424, State: Initialized, Role: FOLLOWER
I20260811 22:22:10.143267 11031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18619927769b4f1aa4d1323ee9d66424" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46083 } }
I20260811 22:22:10.143414 11031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:10.143483 11031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:10.143599 11031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:10.144605 11031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18619927769b4f1aa4d1323ee9d66424" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46083 } }
I20260811 22:22:10.144958 11031 leader_election.cc:302] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18619927769b4f1aa4d1323ee9d66424; no voters:
I20260811 22:22:10.145275 11031 leader_election.cc:288] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:22:10.145391 11039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:10.145637 11039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [term 1 LEADER]: Becoming Leader. State: Replica: 18619927769b4f1aa4d1323ee9d66424, State: Running, Role: LEADER
I20260811 22:22:10.146021 11039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18619927769b4f1aa4d1323ee9d66424" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46083 } }
I20260811 22:22:10.146585 11031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:10.147898 11044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18619927769b4f1aa4d1323ee9d66424. Latest consensus state: current_term: 1 leader_uuid: "18619927769b4f1aa4d1323ee9d66424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18619927769b4f1aa4d1323ee9d66424" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46083 } } }
I20260811 22:22:10.147936 11040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18619927769b4f1aa4d1323ee9d66424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18619927769b4f1aa4d1323ee9d66424" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 46083 } } }
I20260811 22:22:10.148027 11044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:10.148049 11040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:10.148473 11055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:10.152073 11055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:10.158479 11055 catalog_manager.cc:1347] Generated new cluster ID: 7b5364582abe47428cc284cae0cc0730
I20260811 22:22:10.158571 11055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:22:10.183933 11055 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:22:10.185040 11055 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:22:10.204221 11055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18619927769b4f1aa4d1323ee9d66424: Generated new TSK 0
I20260811 22:22:10.205248 11055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:22:10.220553 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46083
--builtin_ntp_servers=127.26.125.20:41759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 22:22:10.343067 11077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:10.343375 11077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:10.343422 11077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:10.347604 11077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:10.347671 11077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:10.347728 11077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:22:10.352756 11077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:10.354035 11077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:10.355659 11077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:10.365697 11088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:10.365689 11087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:10.365985 11077 server_base.cc:1034] running on GCE node
W20260811 22:22:10.366122 11093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:10.366640 11077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:10.367370 11077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:10.368548 11077 hybrid_clock.cc:648] HybridClock initialized: now 1786486930368532 us; error 30 us; skew 500 ppm
I20260811 22:22:10.370669 11077 webserver.cc:460] Webserver started at http://127.26.125.1:39555/ using document root <none> and password file <none>
I20260811 22:22:10.371330 11077 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:10.371452 11077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:10.371671 11077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:10.373984 11077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "99bbbd7d7c634da4aca343d280700f75"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "eb8fa20a2465fa8599cca50e722e2c83"
server_key_iv: "ebf72d81952a38d5582dd5e42abfdfe6"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.374584 11077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "99bbbd7d7c634da4aca343d280700f75"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "eb8fa20a2465fa8599cca50e722e2c83"
server_key_iv: "ebf72d81952a38d5582dd5e42abfdfe6"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.378738 11077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:22:10.381735 11101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.383246 11077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:10.383407 11077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "99bbbd7d7c634da4aca343d280700f75"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "eb8fa20a2465fa8599cca50e722e2c83"
server_key_iv: "ebf72d81952a38d5582dd5e42abfdfe6"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.383569 11077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:10.410478 11077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:10.411353 11077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:10.411635 11077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:10.412292 11077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:10.413352 11077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:10.413403 11077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.413460 11077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:10.413496 11077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.422433 11077 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:38713
I20260811 22:22:10.422457 11316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:38713 every 8 connection(s)
I20260811 22:22:10.423486 11077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:22:10.428979 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 11077
I20260811 22:22:10.429118 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:22:10.429414 27124 external_mini_cluster.cc:1428] Setting key c1a588200e4fd0afb3e68f24580406a9
I20260811 22:22:10.431376 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46083
--builtin_ntp_servers=127.26.125.20:41759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:22:10.433342 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.425517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:59365 (local address 127.26.125.62:46083)
0811 22:22:10.425708 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:10.425713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:10.426909 (+ 1196us) server_negotiation.cc:407] Connection header received
0811 22:22:10.428345 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:10.428347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:10.428397 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:10.428495 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:10.429562 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.430117 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.431346 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.431606 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.431701 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:10.432799 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:10.432803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:10.432806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:22:10.432819 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:10.432840 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:22:10.432849 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:10.432915 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:10.433131 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 22:22:10.433195 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:22:10.434243 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.426000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:46083 (local address 127.26.125.1:59365)
0811 22:22:10.426746 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.426791 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:10.428142 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:10.428705 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:10.428713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:10.428826 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:10.429404 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:10.429421 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.430259 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.430262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:10.431230 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:10.431240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.431756 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.431761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:10.431938 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:10.432534 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:22:10.432536 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:10.432580 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:22:10.432652 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:10.432951 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:10.432963 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:10.432972 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:22:10.433128 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 22:22:10.433386 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260811 22:22:10.435590 11317 heartbeater.cc:347] Connected to a master server at 127.26.125.62:46083
I20260811 22:22:10.435894 11317 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:10.436619 11317 heartbeater.cc:511] Master 127.26.125.62:46083 requested a full tablet report, sending...
I20260811 22:22:10.438340 10964 ts_manager.cc:194] Registered new tserver with Master: 99bbbd7d7c634da4aca343d280700f75 (127.26.125.1:38713)
I20260811 22:22:10.439651 10964 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.1:59365
W20260811 22:22:10.552888 11325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:10.553212 11325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:10.553256 11325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:10.557454 11325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:10.557529 11325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:10.557586 11325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:22:10.562557 11325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:10.563905 11325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:10.565428 11325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:10.576094 11341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:10.576109 11336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:10.576095 11345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:10.576149 11325 server_base.cc:1034] running on GCE node
I20260811 22:22:10.576871 11325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:10.577569 11325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:10.578758 11325 hybrid_clock.cc:648] HybridClock initialized: now 1786486930578724 us; error 48 us; skew 500 ppm
I20260811 22:22:10.580842 11325 webserver.cc:460] Webserver started at http://127.26.125.2:41381/ using document root <none> and password file <none>
I20260811 22:22:10.581450 11325 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:10.581517 11325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:10.581698 11325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:10.584189 11325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "c4576e768aeb48eb8286d149d8bd118b"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "0cfd95fd2cfc34158fa975e8fc0ee04a"
server_key_iv: "49a5d18661360a0548563a3e7a08586f"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.584792 11325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "c4576e768aeb48eb8286d149d8bd118b"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "0cfd95fd2cfc34158fa975e8fc0ee04a"
server_key_iv: "49a5d18661360a0548563a3e7a08586f"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.589378 11325 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:22:10.592418 11351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.593843 11325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:22:10.593966 11325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "c4576e768aeb48eb8286d149d8bd118b"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "0cfd95fd2cfc34158fa975e8fc0ee04a"
server_key_iv: "49a5d18661360a0548563a3e7a08586f"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.594091 11325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:10.611997 11325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:10.612823 11325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:10.613031 11325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:10.613708 11325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:10.614778 11325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:10.614828 11325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.614916 11325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:10.614948 11325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.623720 11325 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:45615
I20260811 22:22:10.623754 11535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:45615 every 8 connection(s)
I20260811 22:22:10.624702 11325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:22:10.629973 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 11325
I20260811 22:22:10.630159 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:22:10.630527 27124 external_mini_cluster.cc:1428] Setting key 26d7bfd706d61e3fa5835fc2d624ca60
I20260811 22:22:10.632683 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46083
--builtin_ntp_servers=127.26.125.20:41759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:22:10.635036 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.626705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:60565 (local address 127.26.125.62:46083)
0811 22:22:10.626881 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:10.626886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:10.628087 (+ 1201us) server_negotiation.cc:407] Connection header received
0811 22:22:10.629524 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:10.629527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:10.629578 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:10.629658 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:10.631039 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.631618 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.632998 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.633283 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.633394 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:10.634528 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:10.634531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:10.634533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:22:10.634547 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:10.634569 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:22:10.634577 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:10.634638 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:10.634786 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 22:22:10.634884 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:22:10.636108 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.627189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:46083 (local address 127.26.125.2:60565)
0811 22:22:10.627912 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.627957 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:10.629251 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:10.629898 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:10.629910 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:10.630040 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:10.630828 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:10.630882 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.631755 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.631758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:10.632804 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:10.632817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.633444 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.633447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:10.633610 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:10.634241 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:22:10.634243 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:10.634291 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:22:10.634368 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:10.634649 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:10.634658 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:10.634665 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:22:10.634779 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:22:10.635155 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":124,"threads_started":1}
I20260811 22:22:10.637336 11536 heartbeater.cc:347] Connected to a master server at 127.26.125.62:46083
I20260811 22:22:10.637642 11536 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:10.638384 11536 heartbeater.cc:511] Master 127.26.125.62:46083 requested a full tablet report, sending...
I20260811 22:22:10.639743 10964 ts_manager.cc:194] Registered new tserver with Master: c4576e768aeb48eb8286d149d8bd118b (127.26.125.2:45615)
I20260811 22:22:10.640326 10964 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.2:60565
W20260811 22:22:10.752949 11543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:10.753235 11543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:10.753273 11543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:10.757473 11543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:10.757548 11543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:10.757604 11543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:22:10.762424 11543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:10.763604 11543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:10.765023 11543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:10.774932 11554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:10.774901 11556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:10.774966 11558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:10.775056 11543 server_base.cc:1034] running on GCE node
I20260811 22:22:10.775669 11543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:10.776366 11543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:10.777598 11543 hybrid_clock.cc:648] HybridClock initialized: now 1786486930777579 us; error 30 us; skew 500 ppm
I20260811 22:22:10.779680 11543 webserver.cc:460] Webserver started at http://127.26.125.3:42925/ using document root <none> and password file <none>
I20260811 22:22:10.780296 11543 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:10.780393 11543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:10.780615 11543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:10.782816 11543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "693eeb7f2f124d0297ec7c296339cdcb"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "3a274d469df8aa08eb2d4e273ef04031"
server_key_iv: "0df91793edce208a07d404146f552d79"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.783470 11543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "693eeb7f2f124d0297ec7c296339cdcb"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "3a274d469df8aa08eb2d4e273ef04031"
server_key_iv: "0df91793edce208a07d404146f552d79"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.787890 11543 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:22:10.791097 11567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.792541 11543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:22:10.792687 11543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "693eeb7f2f124d0297ec7c296339cdcb"
format_stamp: "Formatted at 2026-08-11 22:22:10 on dist-test-slave-5bn2"
server_key: "3a274d469df8aa08eb2d4e273ef04031"
server_key_iv: "0df91793edce208a07d404146f552d79"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.792812 11543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:10.806103 11543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:10.806998 11543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:10.807204 11543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:10.807879 11543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:10.808969 11543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:10.809021 11543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.809080 11543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:10.809110 11543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.817950 11543 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:37519
I20260811 22:22:10.817996 11783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:37519 every 8 connection(s)
I20260811 22:22:10.819000 11543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:22:10.821287 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 11543
I20260811 22:22:10.821408 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:22:10.821740 27124 external_mini_cluster.cc:1428] Setting key 100d676cb7d28022c107640d14da6a1b
I20260811 22:22:10.829804 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.820962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:48555 (local address 127.26.125.62:46083)
0811 22:22:10.821140 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:10.821145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:10.822647 (+ 1502us) server_negotiation.cc:407] Connection header received
0811 22:22:10.824496 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:10.824499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:10.824561 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:10.824659 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:10.826168 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.826796 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.828088 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.828300 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.828376 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:10.829408 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:10.829411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:10.829413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:22:10.829427 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:10.829447 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:22:10.829454 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:10.829506 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:10.829631 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 22:22:10.829684 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:22:10.830713 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.821497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:46083 (local address 127.26.125.3:48555)
0811 22:22:10.822480 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.822526 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:10.824192 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:10.825160 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:10.825170 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:10.825298 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:10.826023 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:10.826044 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.826965 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.826969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:10.827981 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:10.827992 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.828461 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.828464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:10.828616 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:10.829160 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:22:10.829161 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:10.829204 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:22:10.829274 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:10.829520 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:10.829526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:10.829532 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:22:10.829626 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:22:10.829866 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":147,"threads_started":1}
I20260811 22:22:10.831843 11785 heartbeater.cc:347] Connected to a master server at 127.26.125.62:46083
I20260811 22:22:10.832088 11785 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:10.832630 11785 heartbeater.cc:511] Master 127.26.125.62:46083 requested a full tablet report, sending...
I20260811 22:22:10.833771 10964 ts_manager.cc:194] Registered new tserver with Master: 693eeb7f2f124d0297ec7c296339cdcb (127.26.125.3:37519)
I20260811 22:22:10.834333 10964 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.3:48555
I20260811 22:22:10.838332 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:22:10.840993 11037 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:10.840162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43004 (local address 127.26.125.62:46083)
0811 22:22:10.840303 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:10.840307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:10.840446 (+ 139us) server_negotiation.cc:407] Connection header received
0811 22:22:10.840548 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:10.840550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:10.840590 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:10.840653 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:10.840855 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43004: BlockingRecv error: recv got EOF from 127.0.0.1:43004 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:22:10.841089 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 11077
I20260811 22:22:10.871510 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 11325
I20260811 22:22:10.896108 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 11543
I20260811 22:22:10.919903 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 10903
2026-08-11T22:22:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1029 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T22:22:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:22:10Z Disabled control of system clock
I20260811 22:22:10.961848 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:37501
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:37501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:22:11.080668 11802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:11.080987 11802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:11.081027 11802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:11.085258 11802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:11.085358 11802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:11.085373 11802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:11.085389 11802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:11.085403 11802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:11.090772 11802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:37501
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:37501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:11.092007 11802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:11.093462 11802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:22:11.102249 11802 server_base.cc:1034] running on GCE node
W20260811 22:22:11.102207 11808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.102207 11809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.102207 11813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.103147 11802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:11.104414 11802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:11.105613 11802 hybrid_clock.cc:648] HybridClock initialized: now 1786486931105573 us; error 54 us; skew 500 ppm
I20260811 22:22:11.107803 11802 webserver.cc:460] Webserver started at http://127.26.125.62:35511/ using document root <none> and password file <none>
I20260811 22:22:11.108454 11802 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:11.108537 11802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:11.108762 11802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:11.111193 11802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "2f2c5c60b53049baaa91cea4d22b4090"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "765fa25d1885512e90f2f81bf543fd5e"
server_key_iv: "df62e21ed797f90b96aefca0a0edd9f9"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.111833 11802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "2f2c5c60b53049baaa91cea4d22b4090"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "765fa25d1885512e90f2f81bf543fd5e"
server_key_iv: "df62e21ed797f90b96aefca0a0edd9f9"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.116537 11802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:22:11.119470 11820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.120880 11802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:11.121042 11802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "2f2c5c60b53049baaa91cea4d22b4090"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "765fa25d1885512e90f2f81bf543fd5e"
server_key_iv: "df62e21ed797f90b96aefca0a0edd9f9"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.121166 11802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:11.126111 11802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:11.126771 11802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:11.127005 11802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:11.133762 11802 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:37501
I20260811 22:22:11.133780 11906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:37501 every 8 connection(s)
I20260811 22:22:11.134764 11802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:11.138283 11910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:11.138741 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 11802
I20260811 22:22:11.138907 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:22:11.139200 27124 external_mini_cluster.cc:1428] Setting key 5c75887732af7b04bad8d231df69d774
I20260811 22:22:11.145852 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.140805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.26.125.62:37501)
0811 22:22:11.141566 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:11.141587 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:11.141636 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:22:11.142504 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:11.142531 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:11.143091 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:11.143497 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:11.144244 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:11.144250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:11.144267 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:22:11.144278 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:11.144344 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:22:11.144366 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:11.144519 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:11.144620 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 22:22:11.144905 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":115,"threads_started":1}
I20260811 22:22:11.145893 11910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090: Bootstrap starting.
I20260811 22:22:11.148674 11910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:11.149457 11910 log.cc:824] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:11.151854 11910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090: No bootstrap required, opened a new log
I20260811 22:22:11.155530 11910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2c5c60b53049baaa91cea4d22b4090" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37501 } }
I20260811 22:22:11.155812 11910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:11.155867 11910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2c5c60b53049baaa91cea4d22b4090, State: Initialized, Role: FOLLOWER
I20260811 22:22:11.156319 11910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2c5c60b53049baaa91cea4d22b4090" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37501 } }
I20260811 22:22:11.156462 11910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:11.156504 11910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:11.156582 11910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:11.157773 11910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2c5c60b53049baaa91cea4d22b4090" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37501 } }
I20260811 22:22:11.158167 11910 leader_election.cc:302] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f2c5c60b53049baaa91cea4d22b4090; no voters:
I20260811 22:22:11.158494 11910 leader_election.cc:288] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:22:11.158654 11921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:11.158973 11921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [term 1 LEADER]: Becoming Leader. State: Replica: 2f2c5c60b53049baaa91cea4d22b4090, State: Running, Role: LEADER
I20260811 22:22:11.159363 11921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2c5c60b53049baaa91cea4d22b4090" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37501 } }
I20260811 22:22:11.159878 11910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:11.161078 11927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f2c5c60b53049baaa91cea4d22b4090. Latest consensus state: current_term: 1 leader_uuid: "2f2c5c60b53049baaa91cea4d22b4090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2c5c60b53049baaa91cea4d22b4090" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37501 } } }
I20260811 22:22:11.161216 11927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:11.161535 11924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f2c5c60b53049baaa91cea4d22b4090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2c5c60b53049baaa91cea4d22b4090" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37501 } } }
I20260811 22:22:11.161648 11940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:11.161737 11924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:11.165174 11940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:11.171886 11940 catalog_manager.cc:1347] Generated new cluster ID: 0e7b7db1e0674dae9b0fbd54039e2bbc
I20260811 22:22:11.171978 11940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:22:11.183272 11940 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:22:11.183656 11940 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:22:11.193907 11940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f2c5c60b53049baaa91cea4d22b4090: Generated new TSK 0
I20260811 22:22:11.194711 11940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:22:11.204501 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37501
--builtin_ntp_servers=127.26.125.20:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:22:11.329200 11957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:11.329521 11957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:11.329564 11957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:11.333854 11957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:11.333946 11957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:11.334005 11957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:22:11.339071 11957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:11.340257 11957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:11.341717 11957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:11.352721 11972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.352783 11968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.352835 11957 server_base.cc:1034] running on GCE node
W20260811 22:22:11.352732 11969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.353544 11957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:11.354280 11957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:11.355490 11957 hybrid_clock.cc:648] HybridClock initialized: now 1786486931355455 us; error 48 us; skew 500 ppm
I20260811 22:22:11.357682 11957 webserver.cc:460] Webserver started at http://127.26.125.1:39467/ using document root <none> and password file <none>
I20260811 22:22:11.358302 11957 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:11.358369 11957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:11.358566 11957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:11.361001 11957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "235cc350754a44f3952a9f50b7974f84"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "3037065dc607933376c45b03c2e56d1f"
server_key_iv: "d84a0616ce037273b68e4e84b47cbf80"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.361637 11957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "235cc350754a44f3952a9f50b7974f84"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "3037065dc607933376c45b03c2e56d1f"
server_key_iv: "d84a0616ce037273b68e4e84b47cbf80"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.365852 11957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:22:11.369014 11983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.370546 11957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:22:11.370678 11957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "235cc350754a44f3952a9f50b7974f84"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "3037065dc607933376c45b03c2e56d1f"
server_key_iv: "d84a0616ce037273b68e4e84b47cbf80"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.370817 11957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:11.375890 11957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:11.376586 11957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:11.376768 11957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:11.377353 11957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:11.378393 11957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:11.378465 11957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.378516 11957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:11.378544 11957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.387565 11957 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:44021
I20260811 22:22:11.387596 12183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:44021 every 8 connection(s)
I20260811 22:22:11.388557 11957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:22:11.392884 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 11957
I20260811 22:22:11.393042 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:22:11.393397 27124 external_mini_cluster.cc:1428] Setting key 1a1d2c77ec2db9195cee7129e8cf4735
I20260811 22:22:11.395713 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37501
--builtin_ntp_servers=127.26.125.20:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:22:11.395764 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.390626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:55931 (local address 127.26.125.62:37501)
0811 22:22:11.390785 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:11.390789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:11.392130 (+ 1341us) server_negotiation.cc:407] Connection header received
0811 22:22:11.393813 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:11.393817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:11.393871 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:11.393959 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:11.395272 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:11.395275 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:11.395277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:22:11.395293 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:11.395313 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:22:11.395321 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:11.395392 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:11.395605 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 22:22:11.395649 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:22:11.397010 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.391205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:37501 (local address 127.26.125.1:55931)
0811 22:22:11.391951 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:11.391999 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:11.393525 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:11.394184 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:11.394194 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:11.394307 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:11.394949 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:22:11.394952 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:11.395012 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:22:11.395111 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:11.395402 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:11.395412 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:11.395418 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:22:11.395624 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 22:22:11.395929 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":122,"threads_started":1}
I20260811 22:22:11.398330 12184 heartbeater.cc:347] Connected to a master server at 127.26.125.62:37501
I20260811 22:22:11.398622 12184 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:11.399153 12184 heartbeater.cc:511] Master 127.26.125.62:37501 requested a full tablet report, sending...
I20260811 22:22:11.401028 11849 ts_manager.cc:194] Registered new tserver with Master: 235cc350754a44f3952a9f50b7974f84 (127.26.125.1:44021)
W20260811 22:22:11.514209 12190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:11.514528 12190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:11.514569 12190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:11.518738 12190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:11.518827 12190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:11.518961 12190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:22:11.524380 12190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:11.525678 12190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:11.527346 12190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:11.537871 12202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.537895 12201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.538074 12190 server_base.cc:1034] running on GCE node
W20260811 22:22:11.537871 12204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.538628 12190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:11.539418 12190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:11.540612 12190 hybrid_clock.cc:648] HybridClock initialized: now 1786486931540585 us; error 39 us; skew 500 ppm
I20260811 22:22:11.542779 12190 webserver.cc:460] Webserver started at http://127.26.125.2:41201/ using document root <none> and password file <none>
I20260811 22:22:11.543478 12190 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:11.543553 12190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:11.543741 12190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:11.546088 12190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "10b1393348ba4449847b0f44a9029ced"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "c50eb1e0fa876e36eef1001404d26892"
server_key_iv: "e348735e5608d885fa31fdbcf2a1ff6a"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.546657 12190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "10b1393348ba4449847b0f44a9029ced"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "c50eb1e0fa876e36eef1001404d26892"
server_key_iv: "e348735e5608d885fa31fdbcf2a1ff6a"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.550976 12190 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:22:11.554103 12211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.555600 12190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:11.555728 12190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "10b1393348ba4449847b0f44a9029ced"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "c50eb1e0fa876e36eef1001404d26892"
server_key_iv: "e348735e5608d885fa31fdbcf2a1ff6a"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.555843 12190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:11.560890 12190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:11.561619 12190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:11.561789 12190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:11.562402 12190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:11.563449 12190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:11.563499 12190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.563547 12190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:11.563577 12190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.572167 12190 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:38011
I20260811 22:22:11.572203 12423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:38011 every 8 connection(s)
I20260811 22:22:11.573150 12190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:22:11.574522 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 12190
I20260811 22:22:11.574646 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:22:11.575008 27124 external_mini_cluster.cc:1428] Setting key ef249bcad0ad441cc4db2a3e2ef842b8
I20260811 22:22:11.577147 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37501
--builtin_ntp_servers=127.26.125.20:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:22:11.580027 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.575315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:39587 (local address 127.26.125.62:37501)
0811 22:22:11.575519 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:11.575524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:11.576869 (+ 1345us) server_negotiation.cc:407] Connection header received
0811 22:22:11.578265 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:11.578268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:11.578330 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:11.578406 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:11.579567 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:11.579571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:11.579574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:22:11.579588 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:11.579612 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:22:11.579620 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:11.579701 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:11.579827 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 22:22:11.579875 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:22:11.581038 12427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.575837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:37501 (local address 127.26.125.2:39587)
0811 22:22:11.576691 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:11.576735 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:11.578042 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:11.578587 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:11.578595 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:11.578695 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:11.579288 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:22:11.579290 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:11.579339 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:22:11.579416 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:11.579689 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:11.579697 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:11.579703 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:22:11.579871 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 22:22:11.580238 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":141,"threads_started":1}
I20260811 22:22:11.582350 12424 heartbeater.cc:347] Connected to a master server at 127.26.125.62:37501
I20260811 22:22:11.582616 12424 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:11.583150 12424 heartbeater.cc:511] Master 127.26.125.62:37501 requested a full tablet report, sending...
I20260811 22:22:11.584340 11849 ts_manager.cc:194] Registered new tserver with Master: 10b1393348ba4449847b0f44a9029ced (127.26.125.2:38011)
W20260811 22:22:11.697754 12428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:11.698064 12428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:11.698124 12428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:11.702505 12428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:11.702577 12428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:11.702659 12428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:22:11.707775 12428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:11.708938 12428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:11.710419 12428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:11.720927 12434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.720893 12428 server_base.cc:1034] running on GCE node
W20260811 22:22:11.720907 12438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.720861 12436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.721625 12428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:11.722317 12428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:11.723516 12428 hybrid_clock.cc:648] HybridClock initialized: now 1786486931723477 us; error 50 us; skew 500 ppm
I20260811 22:22:11.725709 12428 webserver.cc:460] Webserver started at http://127.26.125.3:45373/ using document root <none> and password file <none>
I20260811 22:22:11.726325 12428 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:11.726393 12428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:11.726611 12428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:11.728895 12428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "93e160c6edb34c6d8289b1c47e0e2a8e"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "ef6f76c0ee499e8794901ca41bc84de8"
server_key_iv: "c1930c2fdee0049a012872625c743ab3"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.729473 12428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "93e160c6edb34c6d8289b1c47e0e2a8e"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "ef6f76c0ee499e8794901ca41bc84de8"
server_key_iv: "c1930c2fdee0049a012872625c743ab3"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.733525 12428 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:22:11.736466 12450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.737932 12428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:22:11.738068 12428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "93e160c6edb34c6d8289b1c47e0e2a8e"
format_stamp: "Formatted at 2026-08-11 22:22:11 on dist-test-slave-5bn2"
server_key: "ef6f76c0ee499e8794901ca41bc84de8"
server_key_iv: "c1930c2fdee0049a012872625c743ab3"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.738193 12428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:11.743289 12428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:11.744010 12428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:11.744195 12428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:11.744791 12428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:11.745819 12428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:11.745870 12428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.745936 12428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:11.745963 12428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.754962 12428 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:45663
I20260811 22:22:11.754982 12645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:45663 every 8 connection(s)
I20260811 22:22:11.755964 12428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:22:11.765241 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 12428
I20260811 22:22:11.765350 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:22:11.765676 27124 external_mini_cluster.cc:1428] Setting key c5455ceac463b4adbeba368e31e267c2
I20260811 22:22:11.771642 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.758085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:49959 (local address 127.26.125.62:37501)
0811 22:22:11.758235 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:11.758238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:11.759473 (+ 1235us) server_negotiation.cc:407] Connection header received
0811 22:22:11.760993 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:11.760996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:11.761044 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:11.761123 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:11.771144 (+ 10021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:11.771148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:11.771150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:22:11.771165 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:11.771189 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:22:11.771198 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:11.771280 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:11.771440 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:22:11.771493 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:22:11.772722 12648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.758518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:37501 (local address 127.26.125.3:49959)
0811 22:22:11.759324 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:11.759367 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:11.760760 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:11.761351 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:11.761360 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:11.761469 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:11.770672 (+ 9203us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:22:11.770675 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:11.770742 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:22:11.770938 (+ 196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:11.771276 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:11.771290 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:11.771299 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:22:11.771506 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:22:11.771866 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":121,"threads_started":1}
I20260811 22:22:11.774088 12646 heartbeater.cc:347] Connected to a master server at 127.26.125.62:37501
I20260811 22:22:11.774346 12646 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:11.774806 12646 heartbeater.cc:511] Master 127.26.125.62:37501 requested a full tablet report, sending...
I20260811 22:22:11.776054 11849 ts_manager.cc:194] Registered new tserver with Master: 93e160c6edb34c6d8289b1c47e0e2a8e (127.26.125.3:45663)
I20260811 22:22:11.780567 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:22:11.783528 11918 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:22:11.782684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.26.125.62:37501)
0811 22:22:11.782804 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:11.782807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:11.782948 (+ 141us) server_negotiation.cc:407] Connection header received
0811 22:22:11.783081 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:11.783083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:11.783120 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:11.783191 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:11.783413 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51854: BlockingRecv error: recv got EOF from 127.0.0.1:51854 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:22:11.783764 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 11957
I20260811 22:22:11.806262 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 12190
I20260811 22:22:11.832476 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 12428
I20260811 22:22:11.853667 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 11802
2026-08-11T22:22:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (935 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:22:11 dist-test-slave-5bn2 krb5kdc[12659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:22:11 dist-test-slave-5bn2 krb5kdc[12659](info): set up 2 sockets
Aug 11 22:22:11 dist-test-slave-5bn2 krb5kdc[12659](info): commencing operation
krb5kdc: starting...
W20260811 22:22:13.074002 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:22:13 dist-test-slave-5bn2 krb5kdc[12659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:22:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:22:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:22:13.219132 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41239
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:41239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:13.338425 12683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:13.338762 12683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:13.338824 12683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:13.343006 12683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:13.343096 12683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:13.343123 12683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:13.343144 12683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:13.343163 12683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:13.348827 12683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:41239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:41239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:13.350032 12683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:13.351199 12683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:13.360131 12693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:13.360131 12692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:13.360147 12695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:13.360249 12683 server_base.cc:1034] running on GCE node
I20260811 22:22:13.361039 12683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:13.362231 12683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:13.363490 12683 hybrid_clock.cc:648] HybridClock initialized: now 1786486933363466 us; error 37 us; skew 500 ppm
Aug 11 22:22:13 dist-test-slave-5bn2 krb5kdc[12659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486933, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:13.366416 12683 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:22:13.367594 12683 webserver.cc:460] Webserver started at http://127.26.125.62:43397/ using document root <none> and password file <none>
I20260811 22:22:13.368242 12683 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:13.368328 12683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:13.368561 12683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:13.370743 12683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "113c30b097444766bbe9e71427da18aa"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "af22001ff808d93d604b5d4b9cec603f"
server_key_iv: "212d949e6c33f6c653d4e8d4e138605a"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.371372 12683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "113c30b097444766bbe9e71427da18aa"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "af22001ff808d93d604b5d4b9cec603f"
server_key_iv: "212d949e6c33f6c653d4e8d4e138605a"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.375466 12683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:22:13.378244 12707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:13.379762 12683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:13.379910 12683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "113c30b097444766bbe9e71427da18aa"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "af22001ff808d93d604b5d4b9cec603f"
server_key_iv: "212d949e6c33f6c653d4e8d4e138605a"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.380031 12683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:13.394696 12683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:13.398236 12683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:13.398447 12683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:13.406243 12683 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:41239
I20260811 22:22:13.406533 12800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:41239 every 8 connection(s)
I20260811 22:22:13.407465 12683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:13.410609 12801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:13.416790 12801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa: Bootstrap starting.
I20260811 22:22:13.417687 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 12683
I20260811 22:22:13.417819 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:22:13.418134 27124 external_mini_cluster.cc:1428] Setting key 85082a35d222f3174a617761b6c64a15
I20260811 22:22:13.419979 12801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:13.420722 12801 log.cc:824] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:13.422677 12801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa: No bootstrap required, opened a new log
I20260811 22:22:13.426311 12801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113c30b097444766bbe9e71427da18aa" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41239 } }
I20260811 22:22:13.426601 12801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:13.426663 12801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 113c30b097444766bbe9e71427da18aa, State: Initialized, Role: FOLLOWER
Aug 11 22:22:13 dist-test-slave-5bn2 krb5kdc[12659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:22:13.427173 12801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113c30b097444766bbe9e71427da18aa" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41239 } }
I20260811 22:22:13.427304 12801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:13.427367 12801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:13.427464 12801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:13.428509 12801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113c30b097444766bbe9e71427da18aa" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41239 } }
I20260811 22:22:13.428870 12801 leader_election.cc:302] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 113c30b097444766bbe9e71427da18aa; no voters:
I20260811 22:22:13.429193 12801 leader_election.cc:288] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:22:13.429278 12808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:13.429535 12808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [term 1 LEADER]: Becoming Leader. State: Replica: 113c30b097444766bbe9e71427da18aa, State: Running, Role: LEADER
I20260811 22:22:13.429903 12808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113c30b097444766bbe9e71427da18aa" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41239 } }
I20260811 22:22:13.430655 12801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:13.431983 12810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 113c30b097444766bbe9e71427da18aa. Latest consensus state: current_term: 1 leader_uuid: "113c30b097444766bbe9e71427da18aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113c30b097444766bbe9e71427da18aa" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41239 } } }
I20260811 22:22:13.432027 12809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "113c30b097444766bbe9e71427da18aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113c30b097444766bbe9e71427da18aa" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 41239 } } }
I20260811 22:22:13.432118 12810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:13.432140 12809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:13.434190 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:13.419422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.26.125.62:41239)
0811 22:22:13.419878 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:13.419898 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:13.419936 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:22:13.420656 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:13.420676 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:13.421071 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:13.421402 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:13.422222 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.423321 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.424173 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.424658 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.424809 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:13.427548 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:13.427555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:13.427575 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:13.427608 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:13.429478 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:13.430000 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:13.430006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:13.430013 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:13.430176 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:13.430515 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:13.430518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:13.430519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:13.431058 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:13.431266 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:13.433018 (+ 1752us) server_negotiation.cc:299] Negotiation successful
0811 22:22:13.433358 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":129,"threads_started":1}
W20260811 22:22:13.436260 12833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:22:13.436426 12833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:22:13.436527 12824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:13.437458 12824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:13.443229 12824 catalog_manager.cc:1347] Generated new cluster ID: b6b3d5d323b3423ca7a2f915841126eb
I20260811 22:22:13.443310 12824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:22:13.452235 12824 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:22:13.453495 12824 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:22:13.459231 12824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa: Generated new TSK 0
I20260811 22:22:13.460014 12824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:22:13.493484 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41239
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:13.615909 12845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:13.616211 12845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:13.616256 12845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:13.620352 12845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:13.620435 12845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:13.620524 12845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:22:13.626077 12845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:13.627272 12845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:13.628325 12845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:13.638641 12856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:13.638641 12852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:13.638756 12845 server_base.cc:1034] running on GCE node
W20260811 22:22:13.638656 12854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:13.639411 12845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:13.640074 12845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:13.641263 12845 hybrid_clock.cc:648] HybridClock initialized: now 1786486933641227 us; error 47 us; skew 500 ppm
Aug 11 22:22:13 dist-test-slave-5bn2 krb5kdc[12659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486933, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:13.644482 12845 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:22:13.645503 12845 webserver.cc:460] Webserver started at http://127.26.125.1:40521/ using document root <none> and password file <none>
I20260811 22:22:13.646109 12845 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:13.646183 12845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:13.646420 12845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:13.648618 12845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "2ac13a30b570ffb7973dca2372cda6dd"
server_key_iv: "a2b1edbe0d7337d932ac17b27e0d0c63"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.649194 12845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "2ac13a30b570ffb7973dca2372cda6dd"
server_key_iv: "a2b1edbe0d7337d932ac17b27e0d0c63"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.653328 12845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:22:13.656068 12868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:13.657388 12845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:13.657510 12845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "2ac13a30b570ffb7973dca2372cda6dd"
server_key_iv: "a2b1edbe0d7337d932ac17b27e0d0c63"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.657611 12845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:13.699865 12845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:13.703769 12845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:13.703974 12845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:13.704629 12845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:13.705578 12845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:13.705631 12845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:13.705690 12845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:13.705716 12845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:13.714401 12845 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:41627
I20260811 22:22:13.714485 13059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:41627 every 8 connection(s)
I20260811 22:22:13.715406 12845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:22:13.722433 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 12845
I20260811 22:22:13.722563 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:22:13.722862 27124 external_mini_cluster.cc:1428] Setting key 00eb101a9f5ad59dbd17e00958e78cf7
Aug 11 22:22:13 dist-test-slave-5bn2 krb5kdc[12659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486933, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:22:13.730119 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:13.717421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:54625 (local address 127.26.125.62:41239)
0811 22:22:13.717590 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:13.717594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:13.718548 (+ 954us) server_negotiation.cc:407] Connection header received
0811 22:22:13.720040 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:13.720043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:13.720102 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:13.720181 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:13.721457 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.721993 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.723233 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.723501 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.723606 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:13.726242 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:13.726247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:13.726250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:13.726283 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:13.727762 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:13.728295 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:13.728299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:13.728301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:13.728399 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:13.728795 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:13.728798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:13.728800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:13.729035 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:13.729130 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:13.729756 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 22:22:13.729928 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:22:13.731137 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:13.717829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41239 (local address 127.26.125.1:54625)
0811 22:22:13.718351 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:13.718431 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:13.719807 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:13.720383 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:13.720390 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:13.720772 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:13.721299 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:13.721312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.722114 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.722116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:13.723111 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:13.723122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.723672 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.723676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:13.723886 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:13.724541 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:13.724558 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:13.726044 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:13.727893 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:13.727898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:13.727914 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:13.728197 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:13.728522 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:13.728525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:13.728526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:13.728702 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:13.729158 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:13.729164 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:13.729486 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 22:22:13.729681 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 22:22:13.730075 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":122,"threads_started":1}
I20260811 22:22:13.732673 13062 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41239
I20260811 22:22:13.732957 13062 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:13.733498 13062 heartbeater.cc:511] Master 127.26.125.62:41239 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:22:13.735333 12742 ts_manager.cc:194] Registered new tserver with Master: ebb938a5dd4b4e24bcbeed1c70bfb0e1 (127.26.125.1:41627)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:22:13.736774 12742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:54625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:22:13.753952 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41239
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:13.871976 13075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:13.872258 13075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:13.872305 13075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:13.876358 13075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:13.876430 13075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:13.876488 13075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:22:13.883744 13075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:13.885246 13075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:13.886790 13075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:13.897586 13083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:13.897748 13088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:13.897648 13084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:13.897878 13075 server_base.cc:1034] running on GCE node
I20260811 22:22:13.898381 13075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:13.899156 13075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:13.900315 13075 hybrid_clock.cc:648] HybridClock initialized: now 1786486933900285 us; error 55 us; skew 500 ppm
Aug 11 22:22:13 dist-test-slave-5bn2 krb5kdc[12659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486933, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:13.903380 13075 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:22:13.904410 13075 webserver.cc:460] Webserver started at http://127.26.125.2:45031/ using document root <none> and password file <none>
I20260811 22:22:13.905009 13075 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:13.905077 13075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:13.905287 13075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:13.907577 13075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "913d28eca9024d159fb21d0cfe73be96"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "5afd5d5d60c5359c6a9f7544abf95d6a"
server_key_iv: "2398c423656986b1f67901e4976cba75"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.908149 13075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "913d28eca9024d159fb21d0cfe73be96"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "5afd5d5d60c5359c6a9f7544abf95d6a"
server_key_iv: "2398c423656986b1f67901e4976cba75"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.912181 13075 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:22:13.914795 13101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:13.916131 13075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:22:13.916257 13075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "913d28eca9024d159fb21d0cfe73be96"
format_stamp: "Formatted at 2026-08-11 22:22:13 on dist-test-slave-5bn2"
server_key: "5afd5d5d60c5359c6a9f7544abf95d6a"
server_key_iv: "2398c423656986b1f67901e4976cba75"
server_key_version: "encryptionkey@0"
I20260811 22:22:13.916373 13075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:13.926403 13075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:13.929394 13075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:13.929613 13075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:13.930236 13075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:13.931196 13075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:13.931257 13075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:13.931317 13075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:13.931342 13075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:13.940017 13075 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:42467
I20260811 22:22:13.940088 13291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:42467 every 8 connection(s)
I20260811 22:22:13.940968 13075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:22:13.941938 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 13075
I20260811 22:22:13.942046 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:22:13.942320 27124 external_mini_cluster.cc:1428] Setting key 70d777774aef1fb640b55f6e81d37740
Aug 11 22:22:13 dist-test-slave-5bn2 krb5kdc[12659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486933, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:22:13.955703 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:13.943165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:34445 (local address 127.26.125.62:41239)
0811 22:22:13.943298 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:13.943302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:13.944171 (+ 869us) server_negotiation.cc:407] Connection header received
0811 22:22:13.945597 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:13.945600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:13.945657 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:13.945738 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:13.947063 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.947777 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.948982 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.949311 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.949422 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:13.952231 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:13.952234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:13.952236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:13.952262 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:13.953576 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:13.954138 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:13.954141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:13.954142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:13.954222 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:13.954610 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:13.954613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:13.954614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:13.954789 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:13.954906 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:13.955429 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 22:22:13.955538 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:22:13.956828 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:13.943484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41239 (local address 127.26.125.2:34445)
0811 22:22:13.944033 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:13.944079 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:13.945385 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:13.945981 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:13.945989 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:13.946361 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:13.946917 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:13.946934 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.947889 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.947891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:13.948852 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:13.948864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:13.949451 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:13.949453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:13.949618 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:13.950256 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:13.950273 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:13.952055 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:13.953699 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:13.953705 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:13.953719 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:13.954025 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:13.954327 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:13.954330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:13.954332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:13.954526 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:13.954905 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:13.954912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:13.955187 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 22:22:13.955351 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:22:13.955745 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":141,"threads_started":1}
I20260811 22:22:13.958086 13293 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41239
I20260811 22:22:13.958360 13293 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:13.958935 13293 heartbeater.cc:511] Master 127.26.125.62:41239 requested a full tablet report, sending...
I20260811 22:22:13.960127 12742 ts_manager.cc:194] Registered new tserver with Master: 913d28eca9024d159fb21d0cfe73be96 (127.26.125.2:42467)
I20260811 22:22:13.960937 12742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:34445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:22:13.971925 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41239
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:14.089027 13301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:14.089339 13301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:14.089385 13301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:14.093629 13301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:14.093700 13301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:14.093767 13301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:22:14.099414 13301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:41239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:22:14.100646 13301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:14.101790 13301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:14.112010 13313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:14.111991 13311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:14.111991 13316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:14.112100 13301 server_base.cc:1034] running on GCE node
I20260811 22:22:14.112749 13301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:14.113451 13301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:14.114624 13301 hybrid_clock.cc:648] HybridClock initialized: now 1786486934114600 us; error 40 us; skew 500 ppm
Aug 11 22:22:14 dist-test-slave-5bn2 krb5kdc[12659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486934, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:14.117564 13301 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:22:14.118556 13301 webserver.cc:460] Webserver started at http://127.26.125.3:38485/ using document root <none> and password file <none>
I20260811 22:22:14.119222 13301 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:14.119299 13301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:14.119550 13301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:14.121901 13301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "dbfdaed406c04a1ca442f8231a0d420c"
format_stamp: "Formatted at 2026-08-11 22:22:14 on dist-test-slave-5bn2"
server_key: "567875d535706a556837e2f93278e5ef"
server_key_iv: "d0e8bfdf47e44ac95497c23469ef0c77"
server_key_version: "encryptionkey@0"
I20260811 22:22:14.122541 13301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "dbfdaed406c04a1ca442f8231a0d420c"
format_stamp: "Formatted at 2026-08-11 22:22:14 on dist-test-slave-5bn2"
server_key: "567875d535706a556837e2f93278e5ef"
server_key_iv: "d0e8bfdf47e44ac95497c23469ef0c77"
server_key_version: "encryptionkey@0"
I20260811 22:22:14.126672 13301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:22:14.129362 13330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:14.130757 13301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:22:14.130919 13301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "dbfdaed406c04a1ca442f8231a0d420c"
format_stamp: "Formatted at 2026-08-11 22:22:14 on dist-test-slave-5bn2"
server_key: "567875d535706a556837e2f93278e5ef"
server_key_iv: "d0e8bfdf47e44ac95497c23469ef0c77"
server_key_version: "encryptionkey@0"
I20260811 22:22:14.131035 13301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:14.142742 13301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:14.145880 13301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:14.146081 13301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:14.146692 13301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:14.147624 13301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:14.147675 13301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:14.147732 13301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:14.147760 13301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:14.156145 13301 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:39835
I20260811 22:22:14.156172 13521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:39835 every 8 connection(s)
I20260811 22:22:14.157122 13301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:22:14.159682 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 13301
I20260811 22:22:14.159827 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:22:14.160200 27124 external_mini_cluster.cc:1428] Setting key 7c525fff1f5a407f421dc8d31852cfc5
Aug 11 22:22:14 dist-test-slave-5bn2 krb5kdc[12659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486934, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:22:14.172562 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.159288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:35769 (local address 127.26.125.62:41239)
0811 22:22:14.159444 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:14.159448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:14.160618 (+ 1170us) server_negotiation.cc:407] Connection header received
0811 22:22:14.162251 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:14.162255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:14.162313 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:14.162394 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:14.163737 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.164263 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.165399 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.165629 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.165770 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.168329 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:14.168332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:14.168334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:14.168360 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:14.169935 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:14.170654 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:14.170658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:14.170659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:14.170760 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:14.171232 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:14.171236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:14.171237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:14.171472 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:14.171572 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:14.172239 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 22:22:14.172365 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:22:14.173728 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.159811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:41239 (local address 127.26.125.3:35769)
0811 22:22:14.160448 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.160496 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:14.162018 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:14.162613 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:14.162621 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:14.163073 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:14.163596 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.163610 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.164380 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.164384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.165280 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.165290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.165811 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.165816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.165993 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.166604 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:14.166621 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:14.168167 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:14.170070 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:14.170076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:14.170106 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:14.170524 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:14.170902 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:14.170905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:14.170907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:14.171095 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:14.171594 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:14.171601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:14.171962 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 22:22:14.172194 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 22:22:14.172570 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":159,"threads_started":1}
I20260811 22:22:14.180050 13522 heartbeater.cc:347] Connected to a master server at 127.26.125.62:41239
I20260811 22:22:14.180337 13522 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:14.180955 13522 heartbeater.cc:511] Master 127.26.125.62:41239 requested a full tablet report, sending...
I20260811 22:22:14.182178 12742 ts_manager.cc:194] Registered new tserver with Master: dbfdaed406c04a1ca442f8231a0d420c (127.26.125.3:39835)
I20260811 22:22:14.182940 12742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:35769
I20260811 22:22:14.186465 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:22:14.195597 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.188360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.26.125.62:41239)
0811 22:22:14.188504 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:14.188507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:14.188617 (+ 110us) server_negotiation.cc:407] Connection header received
0811 22:22:14.188720 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:14.188722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:14.188759 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:14.188824 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:14.189479 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.189984 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.190776 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.191055 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.191137 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.192431 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:14.192434 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:14.192436 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:14.192492 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:14.193887 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:14.194285 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:14.194289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:14.194292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:14.194381 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:14.194637 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:14.194640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:14.194642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:14.194970 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:14.195112 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:14.195318 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 22:22:14.195438 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:22:14.198529 12742 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:22:14.201618 12742 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:22:14.216960 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.210841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.26.125.2:42467)
0811 22:22:14.211992 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:14.212005 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:14.212044 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:22:14.212118 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:14.212123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:14.212307 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:14.212504 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:14.212766 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.213624 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.215345 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.216191 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.216303 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.216458 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:14.216695 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 22:22:14.216793 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":120,"threads_started":1}
I20260811 22:22:14.216957 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.210649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:42467 (local address 127.0.0.1:59652)
0811 22:22:14.211671 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.211685 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:14.211765 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:14.212514 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:14.212517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:14.212536 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:14.212657 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.212664 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.213740 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.213743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.215220 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.215232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.216327 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.216330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.216473 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.216522 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 22:22:14.216640 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:22:14.216781 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"mutex_wait_us":41,"thread_start_us":113,"threads_started":1}
I20260811 22:22:14.216980 13552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.211314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:41627 (local address 127.0.0.1:60816)
0811 22:22:14.211955 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.211966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:14.212046 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:14.212890 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:14.212893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:14.212913 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:14.213001 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.213007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.214294 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.214297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.215269 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.215278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.216535 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.216539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.216625 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.216649 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:22:14.216715 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:22:14.216787 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":118,"threads_started":1}
I20260811 22:22:14.218185 13553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.211648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60816 (local address 127.26.125.1:41627)
0811 22:22:14.212291 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:14.212308 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:14.212330 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:22:14.212431 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:14.212436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:14.212730 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:14.212909 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:14.213106 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.214152 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.215398 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.216191 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.216303 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.216458 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:14.217930 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0811 22:22:14.218037 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":128,"threads_started":1}
I20260811 22:22:14.218305 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.210521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:39835 (local address 127.0.0.1:49842)
0811 22:22:14.211416 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.211474 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:14.211619 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:14.212284 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:14.212290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:14.212327 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:14.212485 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.212493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.213975 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.213978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.215220 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.215232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.217909 (+ 2677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.217912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.218025 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.218047 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:22:14.218119 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:22:14.218185 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":43,"thread_start_us":95,"threads_started":1}
I20260811 22:22:14.218609 13549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.210667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.26.125.3:39835)
0811 22:22:14.211380 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:14.211393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:14.211766 (+ 373us) server_negotiation.cc:407] Connection header received
0811 22:22:14.211855 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:14.211860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:14.212117 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:14.212273 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:14.212624 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.213846 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.216524 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.217609 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.217702 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.217847 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:14.218160 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 22:22:14.218462 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":125,"threads_started":1}
I20260811 22:22:14.220378 13172 tablet_service.cc:1467] Processing CreateTablet for tablet e8eb41801c84450b8711e2b726807f0f (DEFAULT_TABLE table=test-table [id=626bf88cfb794640be10c766e489f098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:14.221638 13172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8eb41801c84450b8711e2b726807f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:14.221432 12939 tablet_service.cc:1467] Processing CreateTablet for tablet e8eb41801c84450b8711e2b726807f0f (DEFAULT_TABLE table=test-table [id=626bf88cfb794640be10c766e489f098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:14.221673 13403 tablet_service.cc:1467] Processing CreateTablet for tablet e8eb41801c84450b8711e2b726807f0f (DEFAULT_TABLE table=test-table [id=626bf88cfb794640be10c766e489f098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:14.222316 12939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8eb41801c84450b8711e2b726807f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:14.222702 13403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8eb41801c84450b8711e2b726807f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:14.228827 13554 tablet_bootstrap.cc:492] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1: Bootstrap starting.
I20260811 22:22:14.230093 13555 tablet_bootstrap.cc:492] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Bootstrap starting.
I20260811 22:22:14.231169 13556 tablet_bootstrap.cc:492] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Bootstrap starting.
I20260811 22:22:14.232503 13554 tablet_bootstrap.cc:654] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:14.233137 13555 tablet_bootstrap.cc:654] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:14.233211 13556 tablet_bootstrap.cc:654] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:14.233645 13554 log.cc:824] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:14.233991 13555 log.cc:824] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:14.234311 13556 log.cc:824] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:14.236016 13554 tablet_bootstrap.cc:492] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1: No bootstrap required, opened a new log
I20260811 22:22:14.236173 13555 tablet_bootstrap.cc:492] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: No bootstrap required, opened a new log
I20260811 22:22:14.236233 13554 ts_tablet_manager.cc:1397] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:22:14.236363 13555 ts_tablet_manager.cc:1397] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:22:14.236707 13556 tablet_bootstrap.cc:492] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: No bootstrap required, opened a new log
I20260811 22:22:14.236892 13556 ts_tablet_manager.cc:1397] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:22:14.240024 13554 raft_consensus.cc:348] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.240276 13554 raft_consensus.cc:374] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:14.240337 13554 raft_consensus.cc:729] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebb938a5dd4b4e24bcbeed1c70bfb0e1, State: Initialized, Role: FOLLOWER
I20260811 22:22:14.240268 13555 raft_consensus.cc:348] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.240542 13555 raft_consensus.cc:374] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:14.240609 13555 raft_consensus.cc:729] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 913d28eca9024d159fb21d0cfe73be96, State: Initialized, Role: FOLLOWER
I20260811 22:22:14.240460 13556 raft_consensus.cc:348] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.240674 13556 raft_consensus.cc:374] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:14.240732 13556 raft_consensus.cc:729] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbfdaed406c04a1ca442f8231a0d420c, State: Initialized, Role: FOLLOWER
I20260811 22:22:14.240867 13554 consensus_queue.cc:260] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.241052 13556 consensus_queue.cc:260] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.241052 13555 consensus_queue.cc:260] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.241631 13062 heartbeater.cc:503] Master 127.26.125.62:41239 was elected leader, sending a full tablet report...
I20260811 22:22:14.242000 13554 ts_tablet_manager.cc:1428] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 22:22:14.242030 13555 ts_tablet_manager.cc:1428] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:22:14.242030 13556 ts_tablet_manager.cc:1428] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:22:14.242369 13293 heartbeater.cc:503] Master 127.26.125.62:41239 was elected leader, sending a full tablet report...
I20260811 22:22:14.242368 13522 heartbeater.cc:503] Master 127.26.125.62:41239 was elected leader, sending a full tablet report...
W20260811 22:22:14.408492 13525 tablet.cc:2369] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:22:14.442209 13294 tablet.cc:2369] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:14.461297 13566 raft_consensus.cc:482] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:22:14.461519 13566 raft_consensus.cc:504] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.462762 13566 leader_election.cc:288] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 913d28eca9024d159fb21d0cfe73be96 (127.26.125.2:42467), ebb938a5dd4b4e24bcbeed1c70bfb0e1 (127.26.125.1:41627)
I20260811 22:22:14.466236 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.463150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:57951 (local address 127.26.125.2:42467)
0811 22:22:14.463267 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:14.463272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:14.463287 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:22:14.463425 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:14.463427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:14.463475 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:14.463538 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:14.463796 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.464290 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.465318 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.465708 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.465814 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.465872 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:14.466041 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 22:22:14.466114 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:22:14.466236 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.463073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:42467 (local address 127.26.125.3:57951)
0811 22:22:14.463194 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.463209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:14.463307 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:14.463558 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:14.463561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:14.463578 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:14.463691 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.463696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.464395 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.464398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.465190 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.465200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.465814 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.465818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.465928 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.465948 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:22:14.466013 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:22:14.466093 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 22:22:14.466513 13553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.463165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:57971 (local address 127.26.125.1:41627)
0811 22:22:14.463287 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:14.463291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:14.463643 (+ 352us) server_negotiation.cc:407] Connection header received
0811 22:22:14.463760 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:14.463763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:14.463803 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:14.463863 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:14.464119 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.464633 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.465607 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.465978 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.466078 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.466131 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:14.466296 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 22:22:14.466375 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:22:14.466645 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.463082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:41627 (local address 127.26.125.3:57971)
0811 22:22:14.463547 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.463558 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:14.463661 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:14.463879 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:14.463882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:14.463896 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:14.464014 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.464019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.464732 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.464735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.465488 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:14.465497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.466078 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.466082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:14.466189 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.466208 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:22:14.466269 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 22:22:14.466460 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":122,"thread_start_us":115,"threads_started":1}
W20260811 22:22:14.466827 13065 tablet.cc:2369] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:14.467039 13209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8eb41801c84450b8711e2b726807f0f" candidate_uuid: "dbfdaed406c04a1ca442f8231a0d420c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "913d28eca9024d159fb21d0cfe73be96" is_pre_election: true
I20260811 22:22:14.467281 12974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8eb41801c84450b8711e2b726807f0f" candidate_uuid: "dbfdaed406c04a1ca442f8231a0d420c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" is_pre_election: true
I20260811 22:22:14.467372 13209 raft_consensus.cc:2454] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbfdaed406c04a1ca442f8231a0d420c in term 0.
I20260811 22:22:14.467551 12974 raft_consensus.cc:2454] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbfdaed406c04a1ca442f8231a0d420c in term 0.
I20260811 22:22:14.467873 13334 leader_election.cc:302] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 913d28eca9024d159fb21d0cfe73be96, dbfdaed406c04a1ca442f8231a0d420c; no voters:
I20260811 22:22:14.468147 13566 raft_consensus.cc:2781] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:22:14.468238 13566 raft_consensus.cc:482] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:22:14.468287 13566 raft_consensus.cc:3037] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:14.469337 13566 raft_consensus.cc:504] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.469676 13566 leader_election.cc:288] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [CANDIDATE]: Term 1 election: Requested vote from peers 913d28eca9024d159fb21d0cfe73be96 (127.26.125.2:42467), ebb938a5dd4b4e24bcbeed1c70bfb0e1 (127.26.125.1:41627)
I20260811 22:22:14.470032 12974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8eb41801c84450b8711e2b726807f0f" candidate_uuid: "dbfdaed406c04a1ca442f8231a0d420c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1"
I20260811 22:22:14.470048 13209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8eb41801c84450b8711e2b726807f0f" candidate_uuid: "dbfdaed406c04a1ca442f8231a0d420c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "913d28eca9024d159fb21d0cfe73be96"
I20260811 22:22:14.470165 12974 raft_consensus.cc:3037] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:14.470172 13209 raft_consensus.cc:3037] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:14.471248 13209 raft_consensus.cc:2454] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbfdaed406c04a1ca442f8231a0d420c in term 1.
I20260811 22:22:14.471307 12974 raft_consensus.cc:2454] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbfdaed406c04a1ca442f8231a0d420c in term 1.
I20260811 22:22:14.471527 13334 leader_election.cc:302] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 913d28eca9024d159fb21d0cfe73be96, dbfdaed406c04a1ca442f8231a0d420c; no voters:
I20260811 22:22:14.471705 13566 raft_consensus.cc:2781] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:14.471944 13566 raft_consensus.cc:686] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 1 LEADER]: Becoming Leader. State: Replica: dbfdaed406c04a1ca442f8231a0d420c, State: Running, Role: LEADER
I20260811 22:22:14.472307 13566 consensus_queue.cc:237] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } }
I20260811 22:22:14.475591 12740 catalog_manager.cc:5495] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c reported cstate change: term changed from 0 to 1, leader changed from <none> to dbfdaed406c04a1ca442f8231a0d420c (127.26.125.3). New cstate: current_term: 1 leader_uuid: "dbfdaed406c04a1ca442f8231a0d420c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfdaed406c04a1ca442f8231a0d420c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 39835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 } health_report { overall_health: UNKNOWN } } }
I20260811 22:22:14.519059 13549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.515519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49844 (local address 127.26.125.3:39835)
0811 22:22:14.515649 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:14.515653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:14.515668 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:22:14.515750 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:14.515752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:14.515810 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:14.515900 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:22:14.516104 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.516600 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.517454 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:14.517658 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:14.517725 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:14.518088 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:22:14.518559 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:22:14.518676 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:14.518774 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 22:22:14.518878 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:22:14.525743 13209 raft_consensus.cc:1261] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 1 FOLLOWER]: Refusing update from remote peer dbfdaed406c04a1ca442f8231a0d420c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:14.525743 12974 raft_consensus.cc:1261] T e8eb41801c84450b8711e2b726807f0f P ebb938a5dd4b4e24bcbeed1c70bfb0e1 [term 1 FOLLOWER]: Refusing update from remote peer dbfdaed406c04a1ca442f8231a0d420c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:14.526577 13578 consensus_queue.cc:1035] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebb938a5dd4b4e24bcbeed1c70bfb0e1" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:14.527000 13566 consensus_queue.cc:1035] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [LEADER]: Connected to new peer: Peer: permanent_uuid: "913d28eca9024d159fb21d0cfe73be96" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:14.546742 12742 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:14.547004 12742 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:14.549806 12742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 113c30b097444766bbe9e71427da18aa: Sending DeleteTablet for 3 replicas of tablet e8eb41801c84450b8711e2b726807f0f
I20260811 22:22:14.550761 13403 tablet_service.cc:1514] Processing DeleteTablet for tablet e8eb41801c84450b8711e2b726807f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:14 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:49842
I20260811 22:22:14.550781 13172 tablet_service.cc:1514] Processing DeleteTablet for tablet e8eb41801c84450b8711e2b726807f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:14 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:59652
I20260811 22:22:14.551358 13596 tablet_replica.cc:330] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: stopping tablet replica
I20260811 22:22:14.551369 12939 tablet_service.cc:1514] Processing DeleteTablet for tablet e8eb41801c84450b8711e2b726807f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:14 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:60816
I20260811 22:22:14.551894 13597 tablet_replica.cc:330] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: stopping tablet replica
I20260811 22:22:14.551884 13596 raft_consensus.cc:2229] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:14.552187 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 12845
I20260811 22:22:14.555184 13596 raft_consensus.cc:2258] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:14.565762 13597 raft_consensus.cc:2229] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:22:14.566301 13597 raft_consensus.cc:2258] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:14.570418 13597 ts_tablet_manager.cc:1905] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:14.570429 13596 ts_tablet_manager.cc:1905] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:14.573330 13596 ts_tablet_manager.cc:1918] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:14.573414 13597 ts_tablet_manager.cc:1918] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:14.573472 13596 log.cc:1196] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/e8eb41801c84450b8711e2b726807f0f
I20260811 22:22:14.573530 13597 log.cc:1196] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/e8eb41801c84450b8711e2b726807f0f
I20260811 22:22:14.573776 13596 ts_tablet_manager.cc:1939] T e8eb41801c84450b8711e2b726807f0f P 913d28eca9024d159fb21d0cfe73be96: Deleting consensus metadata
I20260811 22:22:14.573901 13597 ts_tablet_manager.cc:1939] T e8eb41801c84450b8711e2b726807f0f P dbfdaed406c04a1ca442f8231a0d420c: Deleting consensus metadata
I20260811 22:22:14.575301 12711 catalog_manager.cc:4841] TS 913d28eca9024d159fb21d0cfe73be96 (127.26.125.2:42467): tablet e8eb41801c84450b8711e2b726807f0f (table test-table [id=626bf88cfb794640be10c766e489f098]) successfully deleted
I20260811 22:22:14.575304 12712 catalog_manager.cc:4841] TS dbfdaed406c04a1ca442f8231a0d420c (127.26.125.3:39835): tablet e8eb41801c84450b8711e2b726807f0f (table test-table [id=626bf88cfb794640be10c766e489f098]) successfully deleted
W20260811 22:22:14.576704 12712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:41627 (error 108)
I20260811 22:22:14.576957 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 13075
I20260811 22:22:14.589820 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.582145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:41627 (local address 127.0.0.1:60824)
0811 22:22:14.589567 (+ 7422us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.589698 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:41627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7278}
W20260811 22:22:14.590425 12712 catalog_manager.cc:4584] TS ebb938a5dd4b4e24bcbeed1c70bfb0e1 (127.26.125.1:41627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8eb41801c84450b8711e2b726807f0f: Network error: Client connection negotiation failed: client connection to 127.26.125.1:41627: connect: Connection refused (error 111)
I20260811 22:22:14.597975 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 13301
I20260811 22:22:14.619534 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 12683
2026-08-11T22:22:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2761 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:22:14 dist-test-slave-5bn2 krb5kdc[13605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:22:14 dist-test-slave-5bn2 krb5kdc[13605](info): set up 2 sockets
Aug 11 22:22:14 dist-test-slave-5bn2 krb5kdc[13605](info): commencing operation
krb5kdc: starting...
W20260811 22:22:15.803277 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:22:15 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:22:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:22:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.125.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.125.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/rangeradmin_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/rangeradmin_127.26.125.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.125.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.125.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/rangerlookup_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/rangerlookup_127.26.125.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.125.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.125.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/HTTP_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/HTTP_127.26.125.20@KRBTEST.COM.keytab.
I20260811 22:22:15.994522 27124 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmkw4Fs/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/postgres -l logfile start
2026-08-11 22:22:18.568 UTC [13661] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:22:18.568 UTC [13661] LOG: listening on IPv4 address "127.26.125.20", port 51563
2026-08-11 22:22:18.572 UTC [13661] LOG: listening on Unix socket "/tmp/.s.PGSQL.51563"
2026-08-11 22:22:18.594 UTC [13664] LOG: database system was shut down at 2026-08-11 22:22:17 UTC
2026-08-11 22:22:18.598 UTC [13661] LOG: database system is ready to accept connections
I20260811 22:22:19.798241 27124 mini_postgres.cc:96] Postgres bound to 51563
2026-08-11 22:22:19.808 UTC [13680] FATAL: database "slave" does not exist
127.26.125.20:51563 - accepting connections
I20260811 22:22:19.809202 27124 mini_ranger.cc:161] Starting Ranger...
I20260811 22:22:19.827772 27124 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:22:20.388716 27124 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:22:20.388821 27124 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-admin
2026-08-11 22:22:20,706 [I] DB FLAVOR :POSTGRES
2026-08-11 22:22:20,706 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:22:20,706 [I] Checking connection..
2026-08-11 22:22:20,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:22:21,021 [I] Checking connection passed.
2026-08-11 22:22:21,021 [I] --------- Verifying version history table ---------
2026-08-11 22:22:21,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:22:21,339 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:22:21,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:22:21,627 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:22:21,628 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:22:21,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:22:21.907 UTC [13788] WARNING: there is no transaction in progress
2026-08-11 22:22:21,917 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:22:21,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:22:22,196 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:22:22,196 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:22:22,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:22:22,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:22:22,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5bn2.c.gcp-upstream.internal','N') ;"
2026-08-11 22:22:22,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:22:23,242 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:22:23,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:22:23,550 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:22:23,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:22:23,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:22:24,086 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:22:24,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:22:25.210 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.228 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.250 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.271 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.290 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.312 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.376 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.385 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.403 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.415 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.428 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.444 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.455 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.462 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.469 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:25.476 UTC [13959] WARNING: there is no transaction in progress
2026-08-11 22:22:26,231 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:22:26,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:22:26,477 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:22:26,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:22:26,758 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:22:26,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:22:27,038 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:22:27,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:22:27,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:22:27,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5bn2.c.gcp-upstream.internal';"
2026-08-11 22:22:27,825 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:22:27,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:22:28,102 [I] DB_PATCHES have already been applied
I20260811 22:22:28.108153 27124 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmkw4Fs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:22:28.108242 27124 mini_ranger.cc:193] Using host: 127.26.125.20
I20260811 22:22:28.113498 27124 mini_ranger.cc:227] Ranger admin URL: http://127.26.125.20:36155
Aug 11, 2026 10:22:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:22:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:22:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:22:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:22:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:22:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:22:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:22:29.370167 27124 mini_ranger.cc:160] Time spent starting Ranger: real 9.561s user 0.000s sys 0.015s
Aug 11, 2026 10:22:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:22:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:22:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:22:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:22:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:22:52.751157 27124 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.125.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.125.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/rangerkms_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/rangerkms_127.26.125.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.125.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.125.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.125.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/HTTP_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.125.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/HTTP_127.26.125.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:22:52 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486972, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:22:52.858834 27124 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:22:52.871927 27124 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:22:53.741811 27124 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:22:53.741907 27124 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms
I20260811 22:22:53.741986 27124 mini_ranger_kms.cc:227] Using postgres at 127.26.125.20:51563
2026-08-11 22:22:54,491 [I] DB FLAVOR :POSTGRES
2026-08-11 22:22:54,492 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:22:54,492 [I] Checking connection
2026-08-11 22:22:54,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:22:54,803 [I] connection success
2026-08-11 22:22:54,803 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:22:54,804 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:22:54,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:22:55,100 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:22:55,100 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:22:55,101 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:22:55,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:51563/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:22:55,398 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:22:55.929411 27124 mini_ranger_kms.cc:310] Created kms service
I20260811 22:22:56.173692 27124 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:22:56.221901 27124 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:22:56 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486972, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:22:56.650913 27124 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:22:56.651101 27124 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:22:56.651137 27124 mini_ranger_kms.cc:240] Using host: 127.26.125.20
I20260811 22:22:56.656625 27124 mini_ranger_kms.cc:276] Ranger KMS PID: 14268
I20260811 22:22:56.656842 27124 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.125.20:39063
Aug 11, 2026 10:22:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:22:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:22:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:22:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:22:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:22:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:22:59 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486979, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.125.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:59.292294 27124 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.433s user 0.027s sys 0.429s
I20260811 22:22:59.292469 27124 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:22:59.292614 27124 mini_ranger_kms.cc:401] 127.26.125.20:39063/kms/v1/keys
Aug 11 22:22:59 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486979, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.125.20@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
Aug 11 22:22:59 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486979, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.125.20@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
Aug 11 22:23:02 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:23:02.330704 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:35303
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:35303
--ranger_config_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:39063/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:02.468814 14354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:02.469120 14354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:02.469162 14354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:02.473351 14354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:02.473430 14354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:02.473449 14354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:02.473464 14354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:02.473479 14354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:02.479403 14354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.125.20:39063/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:35303
--ranger_config_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:35303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:02.480641 14354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:02.481737 14354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:02.491667 14367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:02.491669 14366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:02.491643 14369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:02.492022 14354 server_base.cc:1034] running on GCE node
I20260811 22:23:02.492681 14354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:02.493930 14354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:02.495097 14354 hybrid_clock.cc:648] HybridClock initialized: now 1786486982495078 us; error 31 us; skew 500 ppm
Aug 11 22:23:02 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486982, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:02.502012 14354 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:23:02.503140 14354 webserver.cc:460] Webserver started at http://127.26.125.62:38697/ using document root <none> and password file <none>
I20260811 22:23:02.503816 14354 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:02.503885 14354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:02.504129 14354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:23:02 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486982, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:02.569317 14354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "b161b8bff22a44a6aefe301c031dfa48"
format_stamp: "Formatted at 2026-08-11 22:23:02 on dist-test-slave-5bn2"
server_key: "8ec6e0c47c0be7d03b9ec498e7aa40bc"
server_key_iv: "9f42966cb23e51feb73ed1baf21b579c"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:02.570175 14354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "b161b8bff22a44a6aefe301c031dfa48"
format_stamp: "Formatted at 2026-08-11 22:23:02 on dist-test-slave-5bn2"
server_key: "8ec6e0c47c0be7d03b9ec498e7aa40bc"
server_key_iv: "9f42966cb23e51feb73ed1baf21b579c"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:02.576100 14354 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:23:02.592346 14379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:02.593945 14354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:23:02.594099 14354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "b161b8bff22a44a6aefe301c031dfa48"
format_stamp: "Formatted at 2026-08-11 22:23:02 on dist-test-slave-5bn2"
server_key: "8ec6e0c47c0be7d03b9ec498e7aa40bc"
server_key_iv: "9f42966cb23e51feb73ed1baf21b579c"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:02.594254 14354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:02.609359 14354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:02.612937 14354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:02.613184 14354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:02.623638 14354 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:35303
I20260811 22:23:02.623731 14482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:35303 every 8 connection(s)
I20260811 22:23:02.624883 14354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:02.629029 14484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:02.630999 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 14354
I20260811 22:23:02.631141 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:23:02.631456 27124 external_mini_cluster.cc:1428] Setting key a4eccaee5621cdfa11b4eeb2cd806a96
I20260811 22:23:02.637372 14484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48: Bootstrap starting.
Aug 11 22:23:02 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:02.640755 14484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:02.641948 14484 log.cc:824] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:02.644933 14484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48: No bootstrap required, opened a new log
I20260811 22:23:02.647846 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:02.633069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53262 (local address 127.26.125.62:35303)
0811 22:23:02.633552 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:02.633576 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:02.633616 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:23:02.634383 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:02.634403 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:02.634816 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:02.635217 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:02.636091 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:02.637153 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:02.638061 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:02.638576 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:02.638728 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:02.641312 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:02.641322 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:02.641339 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:02.641370 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:02.644124 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:02.644737 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:02.644743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:02.644750 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:02.644905 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:02.645225 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:02.645229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:02.645230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:02.645822 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:02.646033 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:02.646347 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 22:23:02.646670 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":158,"threads_started":1}
I20260811 22:23:02.649511 14484 raft_consensus.cc:348] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b161b8bff22a44a6aefe301c031dfa48" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 35303 } }
I20260811 22:23:02.649777 14484 raft_consensus.cc:374] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:02.649855 14484 raft_consensus.cc:729] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b161b8bff22a44a6aefe301c031dfa48, State: Initialized, Role: FOLLOWER
I20260811 22:23:02.650378 14484 consensus_queue.cc:260] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b161b8bff22a44a6aefe301c031dfa48" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 35303 } }
I20260811 22:23:02.650530 14484 raft_consensus.cc:388] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:02.650614 14484 raft_consensus.cc:482] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:02.650733 14484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:02.651897 14484 raft_consensus.cc:504] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b161b8bff22a44a6aefe301c031dfa48" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 35303 } }
I20260811 22:23:02.652271 14484 leader_election.cc:302] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b161b8bff22a44a6aefe301c031dfa48; no voters:
I20260811 22:23:02.652618 14484 leader_election.cc:288] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:23:02.652769 14494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:02.653034 14494 raft_consensus.cc:686] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [term 1 LEADER]: Becoming Leader. State: Replica: b161b8bff22a44a6aefe301c031dfa48, State: Running, Role: LEADER
I20260811 22:23:02.653446 14494 consensus_queue.cc:237] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b161b8bff22a44a6aefe301c031dfa48" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 35303 } }
I20260811 22:23:02.654152 14484 sys_catalog.cc:564] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:02.655433 14496 sys_catalog.cc:455] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b161b8bff22a44a6aefe301c031dfa48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b161b8bff22a44a6aefe301c031dfa48" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 35303 } } }
I20260811 22:23:02.655576 14496 sys_catalog.cc:458] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:02.655968 14497 sys_catalog.cc:455] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b161b8bff22a44a6aefe301c031dfa48. Latest consensus state: current_term: 1 leader_uuid: "b161b8bff22a44a6aefe301c031dfa48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b161b8bff22a44a6aefe301c031dfa48" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 35303 } } }
I20260811 22:23:02.656056 14497 sys_catalog.cc:458] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:02.657377 14484 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:23:02.657796 14508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:02.661487 14508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:02.674608 14508 catalog_manager.cc:1347] Generated new cluster ID: 7701b05bcc6d40caac76a56967ceb84e
I20260811 22:23:02.674710 14508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:02.693806 14508 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:02.695495 14508 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:02.705826 14508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48: Generated new TSK 0
I20260811 22:23:02.707293 14508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:23:03.501829 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:39063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:35303
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:03.695482 14568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:03.696000 14568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:03.696144 14568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:03.702152 14568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:03.702257 14568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:03.702349 14568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:23:03.711058 14568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.125.20:39063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:03.712651 14568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:03.714169 14568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:03.727044 14578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:03.727020 14577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:03.727381 14568 server_base.cc:1034] running on GCE node
W20260811 22:23:03.727594 14580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:03.728036 14568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:03.728696 14568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:03.729863 14568 hybrid_clock.cc:648] HybridClock initialized: now 1786486983729814 us; error 67 us; skew 500 ppm
Aug 11 22:23:03 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486983, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:03.733961 14568 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:23:03.735350 14568 webserver.cc:460] Webserver started at http://127.26.125.1:44547/ using document root <none> and password file <none>
I20260811 22:23:03.736183 14568 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:03.736287 14568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:03.736649 14568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:23:03 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486983, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:03.755158 14568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "fe0f3d599d714a39b2f7e58d23bad12f"
format_stamp: "Formatted at 2026-08-11 22:23:03 on dist-test-slave-5bn2"
server_key: "af54de3800cec5f49be05e69a238c3f5"
server_key_iv: "942bd60ca5d635bc30b9cc5c99c280ba"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:03.756062 14568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "fe0f3d599d714a39b2f7e58d23bad12f"
format_stamp: "Formatted at 2026-08-11 22:23:03 on dist-test-slave-5bn2"
server_key: "af54de3800cec5f49be05e69a238c3f5"
server_key_iv: "942bd60ca5d635bc30b9cc5c99c280ba"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:03.762305 14568 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:23:03.787719 14589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:03.789839 14568 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260811 22:23:03.790117 14568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "fe0f3d599d714a39b2f7e58d23bad12f"
format_stamp: "Formatted at 2026-08-11 22:23:03 on dist-test-slave-5bn2"
server_key: "af54de3800cec5f49be05e69a238c3f5"
server_key_iv: "942bd60ca5d635bc30b9cc5c99c280ba"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:03.790390 14568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:03.850708 14568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:03.857649 14568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:03.857928 14568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:03.858770 14568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:03.861717 14568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:03.862946 14568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260811 22:23:03.863101 14568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:03.863165 14568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:03.877163 14568 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:41809
I20260811 22:23:03.878489 14568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:03.883520 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 14568
I20260811 22:23:03.883816 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:23:03.884198 27124 external_mini_cluster.cc:1428] Setting key 857ef4122ae4efdeb1ca74438812e9df
I20260811 22:23:03.889710 14795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:41809 every 8 connection(s)
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
Aug 11 22:23:03 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486983, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:03.914151 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:03.895457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:55039 (local address 127.26.125.62:35303)
0811 22:23:03.897643 (+ 2186us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:03.897648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:03.897662 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:23:03.897760 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:03.897763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:03.897815 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:03.897909 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:03.899957 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:03.900847 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:03.902796 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:03.903211 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:03.903332 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:03.908110 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:03.908114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:03.908118 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:03.908153 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:03.910165 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:03.910985 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:03.910990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:03.910992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:03.911107 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:03.911552 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:03.911556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:03.911557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:03.911816 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:03.911926 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:03.913800 (+ 1874us) server_negotiation.cc:299] Negotiation successful
0811 22:23:03.913960 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2097,"thread_start_us":97,"threads_started":1}
I20260811 22:23:03.915313 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:03.895299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:35303 (local address 127.26.125.1:55039)
0811 22:23:03.895950 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:03.896011 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:03.897158 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:03.898257 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:03.898268 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:03.898915 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:03.899741 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:03.899767 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:03.901035 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:03.901039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:03.902639 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:03.902652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:03.903557 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:03.903562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:03.903813 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:03.904591 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:03.904610 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:03.907818 (+ 3208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:03.910340 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:03.910348 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:03.910372 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:03.910778 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:03.911221 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:03.911225 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:03.911226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:03.911434 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:03.911986 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:03.911994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:03.912369 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 22:23:03.912601 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 22:23:03.913000 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":141,"threads_started":1}
I20260811 22:23:03.917181 14798 heartbeater.cc:347] Connected to a master server at 127.26.125.62:35303
I20260811 22:23:03.917471 14798 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:03.918133 14798 heartbeater.cc:511] Master 127.26.125.62:35303 requested a full tablet report, sending...
I20260811 22:23:03.919857 14417 ts_manager.cc:194] Registered new tserver with Master: fe0f3d599d714a39b2f7e58d23bad12f (127.26.125.1:41809)
I20260811 22:23:03.921103 14417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:55039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:23:03.933638 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:39063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:35303
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:23:03 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486983, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:23:04.084429 14813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:04.084743 14813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:04.084789 14813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:04.089051 14813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:04.089126 14813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:04.089186 14813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:23:04.095062 14813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.125.20:39063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:04.096379 14813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:04.097610 14813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:04.111390 14831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:04.111390 14828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:04.111390 14827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:04.112108 14813 server_base.cc:1034] running on GCE node
I20260811 22:23:04.112721 14813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:04.113696 14813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:04.114902 14813 hybrid_clock.cc:648] HybridClock initialized: now 1786486984114877 us; error 32 us; skew 500 ppm
Aug 11 22:23:04 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486984, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:04.119560 14813 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:23:04.120893 14813 webserver.cc:460] Webserver started at http://127.26.125.2:42969/ using document root <none> and password file <none>
I20260811 22:23:04.121757 14813 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:04.121905 14813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:04.122236 14813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:23:04 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486984, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:04.146608 14813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "08686433af3f4ae89e8aa877a6247682"
format_stamp: "Formatted at 2026-08-11 22:23:04 on dist-test-slave-5bn2"
server_key: "86be9cd3bc3220205a9a735010e5a715"
server_key_iv: "b4ce7557da8b6447e2b51e16e8b9928c"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:04.148801 14813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "08686433af3f4ae89e8aa877a6247682"
format_stamp: "Formatted at 2026-08-11 22:23:04 on dist-test-slave-5bn2"
server_key: "86be9cd3bc3220205a9a735010e5a715"
server_key_iv: "b4ce7557da8b6447e2b51e16e8b9928c"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:04.154994 14813 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:23:04.183393 14840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:04.185441 14813 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.001s
I20260811 22:23:04.185602 14813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "08686433af3f4ae89e8aa877a6247682"
format_stamp: "Formatted at 2026-08-11 22:23:04 on dist-test-slave-5bn2"
server_key: "86be9cd3bc3220205a9a735010e5a715"
server_key_iv: "b4ce7557da8b6447e2b51e16e8b9928c"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:04.185775 14813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:04.211674 14813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:04.215862 14813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:04.216173 14813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:04.217202 14813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:04.218505 14813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:04.218623 14813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:04.218740 14813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:04.218811 14813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:04.233987 14813 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:41761
I20260811 22:23:04.235420 14813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:04.244100 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 14813
I20260811 22:23:04.244261 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:23:04.244568 27124 external_mini_cluster.cc:1428] Setting key ac94b6f996180a0a70b0597a3acf8d3f
I20260811 22:23:04.249920 15033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:41761 every 8 connection(s)
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Aug 11 22:23:04 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486984, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:04.277937 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.253971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:48223 (local address 127.26.125.62:35303)
0811 22:23:04.254174 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.254178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.259703 (+ 5525us) server_negotiation.cc:407] Connection header received
0811 22:23:04.261256 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.261261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.261332 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.261390 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:04.263500 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.264361 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.266165 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.266525 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.266645 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.272721 (+ 6076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:04.272726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:04.272729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:04.272770 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:04.274682 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:04.275608 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:04.275614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:04.275616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:04.275737 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:04.276263 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:04.276267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:04.276269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:04.276568 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:04.276661 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.277552 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.277721 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 22:23:04.280212 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.252652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:35303 (local address 127.26.125.2:48223)
0811 22:23:04.259427 (+ 6775us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.259541 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:04.260885 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:04.261918 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:04.261930 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:04.262416 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:04.263302 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.263326 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.264527 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.264530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.266002 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.266015 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.266725 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.266729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.267016 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.267787 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:04.267810 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:04.272426 (+ 4616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:04.274916 (+ 2490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:04.274926 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:04.274953 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:04.275450 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:04.275873 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:04.275877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:04.275879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:04.276134 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:04.276743 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:04.276751 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:04.277201 (+ 450us) client_negotiation.cc:769] Sending connection context
0811 22:23:04.278313 (+ 1112us) client_negotiation.cc:240] Negotiation successful
0811 22:23:04.278803 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6383,"thread_start_us":202,"threads_started":1}
I20260811 22:23:04.282261 15034 heartbeater.cc:347] Connected to a master server at 127.26.125.62:35303
I20260811 22:23:04.282588 15034 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:04.283347 15034 heartbeater.cc:511] Master 127.26.125.62:35303 requested a full tablet report, sending...
I20260811 22:23:04.284987 14417 ts_manager.cc:194] Registered new tserver with Master: 08686433af3f4ae89e8aa877a6247682 (127.26.125.2:41761)
I20260811 22:23:04.285869 14417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:48223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:23:04.291965 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:39063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:35303
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:04.489431 15043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:04.489962 15043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:04.490103 15043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:04.496068 15043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:04.496332 15043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:04.496491 15043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:23:04.505734 15043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.125.20:39063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:04.507637 15043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:04.509258 15043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 11 22:23:04 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486983, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
W20260811 22:23:04.524170 15054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:04.524726 15059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:04.525650 15043 server_base.cc:1034] running on GCE node
W20260811 22:23:04.525933 15053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:04.526731 15043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:04.527676 15043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:04.530928 15043 hybrid_clock.cc:648] HybridClock initialized: now 1786486984530819 us; error 116 us; skew 500 ppm
Aug 11 22:23:04 dist-test-slave-5bn2 krb5kdc[13605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486984, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:04.535389 15043 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:23:04.536746 15043 webserver.cc:460] Webserver started at http://127.26.125.3:38969/ using document root <none> and password file <none>
I20260811 22:23:04.538237 15043 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:04.538405 15043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:04.538738 15043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:23:04 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486984, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:04.571321 15043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "1a4d71b171e34e9288826d90d31b3bf3"
format_stamp: "Formatted at 2026-08-11 22:23:04 on dist-test-slave-5bn2"
server_key: "7440224c105393a9581ef21bd6190e0b"
server_key_iv: "73b6cf4e598d4cf047def51fe214bf2b"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:04.573411 15043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "1a4d71b171e34e9288826d90d31b3bf3"
format_stamp: "Formatted at 2026-08-11 22:23:04 on dist-test-slave-5bn2"
server_key: "7440224c105393a9581ef21bd6190e0b"
server_key_iv: "73b6cf4e598d4cf047def51fe214bf2b"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:04.579669 15043 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 22:23:04.603098 15071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:04.605029 15043 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 22:23:04.605274 15043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "1a4d71b171e34e9288826d90d31b3bf3"
format_stamp: "Formatted at 2026-08-11 22:23:04 on dist-test-slave-5bn2"
server_key: "7440224c105393a9581ef21bd6190e0b"
server_key_iv: "73b6cf4e598d4cf047def51fe214bf2b"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:04.605509 15043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:04.630307 15043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:04.634222 15043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:04.634490 15043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:04.635309 15043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:04.636597 15043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:04.636720 15043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:04.636817 15043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:04.636878 15043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:04.653959 15043 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:42365
I20260811 22:23:04.655714 15043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:04.656572 15254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:42365 every 8 connection(s)
I20260811 22:23:04.660714 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 15043
I20260811 22:23:04.660852 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:23:04.661161 27124 external_mini_cluster.cc:1428] Setting key 5e6a08663a79b9837234d831fc332421
Aug 11 22:23:04 dist-test-slave-5bn2 krb5kdc[13605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486984, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:04.677256 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.661108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:44915 (local address 127.26.125.62:35303)
0811 22:23:04.661243 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.661248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.662492 (+ 1244us) server_negotiation.cc:407] Connection header received
0811 22:23:04.663969 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.663973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.664032 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.664080 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:04.666033 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.666832 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.668621 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.668960 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.669042 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.672628 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:04.672633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:04.672636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:04.672671 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:04.674473 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:04.675240 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:04.675245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:04.675247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:04.675345 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:04.675795 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:04.675799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:04.675801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:04.676068 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:04.676153 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.676941 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.677082 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:23:04.679412 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.659747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:35303 (local address 127.26.125.3:44915)
0811 22:23:04.662315 (+ 2568us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.662387 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:04.663687 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:04.664555 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:04.664566 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:04.665029 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:04.665840 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.665863 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.667052 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.667055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.668466 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.668479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.669268 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.669272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.669509 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.670318 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:04.670342 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:04.672369 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:04.674639 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:04.674646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:04.674669 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:04.675113 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:04.675464 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:04.675466 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:04.675468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:04.675680 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:04.676225 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:04.676233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:04.676618 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 22:23:04.677729 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0811 22:23:04.678149 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2299,"thread_start_us":200,"threads_started":1}
I20260811 22:23:04.681195 15255 heartbeater.cc:347] Connected to a master server at 127.26.125.62:35303
I20260811 22:23:04.681492 15255 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:04.682148 15255 heartbeater.cc:511] Master 127.26.125.62:35303 requested a full tablet report, sending...
I20260811 22:23:04.683532 14417 ts_manager.cc:194] Registered new tserver with Master: 1a4d71b171e34e9288826d90d31b3bf3 (127.26.125.3:42365)
I20260811 22:23:04.684314 14417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:44915
I20260811 22:23:04.687670 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:04.700748 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.690468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.26.125.62:35303)
0811 22:23:04.690589 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.690593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.691256 (+ 663us) server_negotiation.cc:407] Connection header received
0811 22:23:04.691523 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.691526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.691577 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.691625 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:04.692769 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.693533 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.694789 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.695146 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.695244 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.696995 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:04.696999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:04.697002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:04.697038 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:04.698726 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:04.699311 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:04.699315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:04.699317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:04.699410 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:04.699696 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:04.699699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:04.699701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:04.699932 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:04.700067 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.700405 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.700566 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:23:04.704861 14417 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:23:04.709466 14417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:23:04.728299 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.720194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:42365 (local address 127.0.0.1:46210)
0811 22:23:04.722058 (+ 1864us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.722067 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:04.722140 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:04.722935 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:04.722943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:04.722968 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:04.723125 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.723133 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.724895 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.724899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.726222 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.726236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.727643 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.727647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.727807 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.727868 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 22:23:04.727992 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 22:23:04.728129 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1816,"mutex_wait_us":42,"thread_start_us":105,"threads_started":1}
I20260811 22:23:04.728814 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.720690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:41809 (local address 127.0.0.1:35812)
0811 22:23:04.722276 (+ 1586us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.722284 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:04.722371 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:04.723025 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:04.723029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:04.723049 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:04.723153 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.723159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.724504 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.724507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.726950 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.726963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.728155 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.728160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.728260 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.728288 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:23:04.728336 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 22:23:04.728412 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1545,"mutex_wait_us":1059,"thread_start_us":94,"threads_started":1}
I20260811 22:23:04.728931 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.720788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46210 (local address 127.26.125.3:42365)
0811 22:23:04.721443 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.721455 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.722176 (+ 721us) server_negotiation.cc:407] Connection header received
0811 22:23:04.722281 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.722287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.722542 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.722719 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:04.723279 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.724751 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.726412 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.727295 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.727396 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.727576 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.728622 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.728759 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":160,"threads_started":1}
I20260811 22:23:04.729763 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.719979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.26.125.2:41761)
0811 22:23:04.722604 (+ 2625us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.722635 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.722657 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:23:04.722729 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.722735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.722934 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.724214 (+ 1280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:04.724370 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.725358 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.727654 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.728689 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.729379 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.729509 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.729554 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.729628 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2520,"thread_start_us":90,"threads_started":1}
I20260811 22:23:04.729277 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.719856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:41761 (local address 127.0.0.1:41568)
0811 22:23:04.721706 (+ 1850us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.721747 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:04.721856 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:04.723106 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:04.723109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:04.723124 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:04.723232 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.723238 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.725495 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.725498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.726545 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.726554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.728825 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.728829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.729038 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.729063 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:23:04.729117 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 22:23:04.729183 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1688,"mutex_wait_us":1,"thread_start_us":143,"threads_started":1}
I20260811 22:23:04.731195 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.721136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.26.125.1:41809)
0811 22:23:04.721446 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.721454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.722379 (+ 925us) server_negotiation.cc:407] Connection header received
0811 22:23:04.722470 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.722475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.722687 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.722830 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:04.723289 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.724375 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.727119 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.727992 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.730197 (+ 2205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.730383 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.730437 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.730539 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260811 22:23:04.732689 15130 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad9788d03f14ad2a95543972375d420 (DEFAULT_TABLE table=test-table [id=947aa5fddc6e4f2cae09f1d19e99f98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:04.733062 14926 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad9788d03f14ad2a95543972375d420 (DEFAULT_TABLE table=test-table [id=947aa5fddc6e4f2cae09f1d19e99f98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:04.734014 14926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad9788d03f14ad2a95543972375d420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:04.734014 15130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad9788d03f14ad2a95543972375d420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:04.734288 14679 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad9788d03f14ad2a95543972375d420 (DEFAULT_TABLE table=test-table [id=947aa5fddc6e4f2cae09f1d19e99f98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:04.735466 14679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad9788d03f14ad2a95543972375d420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:04.744151 15284 tablet_bootstrap.cc:492] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682: Bootstrap starting.
I20260811 22:23:04.744557 15286 tablet_bootstrap.cc:492] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f: Bootstrap starting.
I20260811 22:23:04.747249 15286 tablet_bootstrap.cc:654] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:04.747776 15284 tablet_bootstrap.cc:654] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:04.748368 15281 tablet_bootstrap.cc:492] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3: Bootstrap starting.
I20260811 22:23:04.748456 15286 log.cc:824] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:04.751894 15284 log.cc:824] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:04.752362 15286 tablet_bootstrap.cc:492] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f: No bootstrap required, opened a new log
I20260811 22:23:04.752588 15286 ts_tablet_manager.cc:1397] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260811 22:23:04.753455 15281 tablet_bootstrap.cc:654] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:04.754427 15281 log.cc:824] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:04.756951 15284 tablet_bootstrap.cc:492] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682: No bootstrap required, opened a new log
I20260811 22:23:04.757290 15284 ts_tablet_manager.cc:1397] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260811 22:23:04.759579 15286 raft_consensus.cc:348] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.760073 15286 raft_consensus.cc:374] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:04.760190 15286 raft_consensus.cc:729] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0f3d599d714a39b2f7e58d23bad12f, State: Initialized, Role: FOLLOWER
I20260811 22:23:04.760859 15286 consensus_queue.cc:260] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.761987 15284 raft_consensus.cc:348] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.762359 15286 ts_tablet_manager.cc:1428] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f: Time spent starting tablet: real 0.010s user 0.005s sys 0.002s
I20260811 22:23:04.762403 15284 raft_consensus.cc:374] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:04.762562 15284 raft_consensus.cc:729] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08686433af3f4ae89e8aa877a6247682, State: Initialized, Role: FOLLOWER
I20260811 22:23:04.762809 14798 heartbeater.cc:503] Master 127.26.125.62:35303 was elected leader, sending a full tablet report...
I20260811 22:23:04.765976 15281 tablet_bootstrap.cc:492] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3: No bootstrap required, opened a new log
I20260811 22:23:04.766301 15281 ts_tablet_manager.cc:1397] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.004s
I20260811 22:23:04.770499 15281 raft_consensus.cc:348] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.770766 15281 raft_consensus.cc:374] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:04.771462 15284 consensus_queue.cc:260] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.772881 15284 ts_tablet_manager.cc:1428] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682: Time spent starting tablet: real 0.015s user 0.002s sys 0.004s
I20260811 22:23:04.773270 15034 heartbeater.cc:503] Master 127.26.125.62:35303 was elected leader, sending a full tablet report...
I20260811 22:23:04.773244 15281 raft_consensus.cc:729] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4d71b171e34e9288826d90d31b3bf3, State: Initialized, Role: FOLLOWER
I20260811 22:23:04.773833 15281 consensus_queue.cc:260] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.775012 15281 ts_tablet_manager.cc:1428] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260811 22:23:04.775422 15255 heartbeater.cc:503] Master 127.26.125.62:35303 was elected leader, sending a full tablet report...
I20260811 22:23:04.790594 15292 raft_consensus.cc:482] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:04.791013 15292 raft_consensus.cc:504] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.792666 15292 leader_election.cc:288] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe0f3d599d714a39b2f7e58d23bad12f (127.26.125.1:41809), 1a4d71b171e34e9288826d90d31b3bf3 (127.26.125.3:42365)
I20260811 22:23:04.798367 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.793129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:43703 (local address 127.26.125.1:41809)
0811 22:23:04.793267 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.793271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.793506 (+ 235us) server_negotiation.cc:407] Connection header received
0811 22:23:04.793723 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.793725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.793777 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.793824 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:04.794344 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.795089 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.796857 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.797481 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.797569 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.797625 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.798131 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.798220 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:23:04.798487 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.793892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:34273 (local address 127.26.125.3:42365)
0811 22:23:04.794024 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.794028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.794297 (+ 269us) server_negotiation.cc:407] Connection header received
0811 22:23:04.794467 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.794470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.794520 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.794580 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:04.795024 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.795720 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.797267 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.797786 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.797875 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.797943 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.798316 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.798388 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:23:04.798959 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.793380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:42365 (local address 127.26.125.2:34273)
0811 22:23:04.794211 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.794224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:04.794361 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:04.794710 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:04.794713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:04.794729 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:04.794908 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.794915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.795904 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.795907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.797138 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.797148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.798046 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.798049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.798186 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.798208 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:23:04.798692 (+ 484us) client_negotiation.cc:240] Negotiation successful
0811 22:23:04.798773 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":42,"thread_start_us":138,"threads_started":1}
I20260811 22:23:04.799417 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.792871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:41809 (local address 127.26.125.2:43703)
0811 22:23:04.793417 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.793433 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:04.793593 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:04.794047 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:04.794050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:04.794072 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:04.794204 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.794212 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.795215 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.795217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.796679 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:04.796691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.797893 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.797896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:04.797985 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.798007 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:23:04.799187 (+ 1180us) client_negotiation.cc:240] Negotiation successful
0811 22:23:04.799264 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":28,"thread_start_us":147,"threads_started":1}
I20260811 22:23:04.799726 15177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad9788d03f14ad2a95543972375d420" candidate_uuid: "08686433af3f4ae89e8aa877a6247682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a4d71b171e34e9288826d90d31b3bf3" is_pre_election: true
I20260811 22:23:04.800117 15177 raft_consensus.cc:2454] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08686433af3f4ae89e8aa877a6247682 in term 0.
I20260811 22:23:04.800750 14719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad9788d03f14ad2a95543972375d420" candidate_uuid: "08686433af3f4ae89e8aa877a6247682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" is_pre_election: true
I20260811 22:23:04.800750 14848 leader_election.cc:302] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08686433af3f4ae89e8aa877a6247682, 1a4d71b171e34e9288826d90d31b3bf3; no voters:
I20260811 22:23:04.801049 14719 raft_consensus.cc:2454] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08686433af3f4ae89e8aa877a6247682 in term 0.
I20260811 22:23:04.801106 15292 raft_consensus.cc:2781] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:04.801183 15292 raft_consensus.cc:482] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:04.801223 15292 raft_consensus.cc:3037] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:04.802639 15292 raft_consensus.cc:504] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.803050 15292 leader_election.cc:288] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [CANDIDATE]: Term 1 election: Requested vote from peers fe0f3d599d714a39b2f7e58d23bad12f (127.26.125.1:41809), 1a4d71b171e34e9288826d90d31b3bf3 (127.26.125.3:42365)
I20260811 22:23:04.803409 14719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad9788d03f14ad2a95543972375d420" candidate_uuid: "08686433af3f4ae89e8aa877a6247682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe0f3d599d714a39b2f7e58d23bad12f"
I20260811 22:23:04.803476 15177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad9788d03f14ad2a95543972375d420" candidate_uuid: "08686433af3f4ae89e8aa877a6247682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a4d71b171e34e9288826d90d31b3bf3"
I20260811 22:23:04.803548 14719 raft_consensus.cc:3037] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:04.803579 15177 raft_consensus.cc:3037] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:04.805011 14719 raft_consensus.cc:2454] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08686433af3f4ae89e8aa877a6247682 in term 1.
I20260811 22:23:04.805011 15177 raft_consensus.cc:2454] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08686433af3f4ae89e8aa877a6247682 in term 1.
I20260811 22:23:04.805375 14848 leader_election.cc:302] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08686433af3f4ae89e8aa877a6247682, 1a4d71b171e34e9288826d90d31b3bf3; no voters:
I20260811 22:23:04.805624 15292 raft_consensus.cc:2781] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:04.805842 15292 raft_consensus.cc:686] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [term 1 LEADER]: Becoming Leader. State: Replica: 08686433af3f4ae89e8aa877a6247682, State: Running, Role: LEADER
I20260811 22:23:04.806246 15292 consensus_queue.cc:237] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } }
I20260811 22:23:04.809696 14417 catalog_manager.cc:5495] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08686433af3f4ae89e8aa877a6247682 (127.26.125.2). New cstate: current_term: 1 leader_uuid: "08686433af3f4ae89e8aa877a6247682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08686433af3f4ae89e8aa877a6247682" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 41761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 } health_report { overall_health: UNKNOWN } } }
I20260811 22:23:04.835878 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.832201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41570 (local address 127.26.125.2:41761)
0811 22:23:04.832319 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:04.832324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:04.832380 (+ 56us) server_negotiation.cc:407] Connection header received
0811 22:23:04.832579 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:04.832581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:04.832643 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:04.832742 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:23:04.832981 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.833484 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.834355 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:04.834582 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:04.834670 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:04.835020 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:23:04.835477 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:23:04.835603 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:04.835667 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 22:23:04.835728 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:23:04.842275 15177 raft_consensus.cc:1261] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 1 FOLLOWER]: Refusing update from remote peer 08686433af3f4ae89e8aa877a6247682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:04.842275 14719 raft_consensus.cc:1261] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f [term 1 FOLLOWER]: Refusing update from remote peer 08686433af3f4ae89e8aa877a6247682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:04.843091 15292 consensus_queue.cc:1035] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe0f3d599d714a39b2f7e58d23bad12f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:04.843401 15299 consensus_queue.cc:1035] T 3ad9788d03f14ad2a95543972375d420 P 08686433af3f4ae89e8aa877a6247682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a4d71b171e34e9288826d90d31b3bf3" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 42365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:04.863660 15306 mvcc.cc:205] Tried to move back new op lower bound from 7317450689906262016 to 7317450689770352640. Current Snapshot: MvccSnapshot[applied={T|T < 7317450689906262016}]
I20260811 22:23:04.863991 15311 mvcc.cc:205] Tried to move back new op lower bound from 7317450689906262016 to 7317450689770352640. Current Snapshot: MvccSnapshot[applied={T|T < 7317450689906262016}]
I20260811 22:23:04.874944 14413 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:04.875137 14413 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:04.877979 14413 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b161b8bff22a44a6aefe301c031dfa48: Sending DeleteTablet for 3 replicas of tablet 3ad9788d03f14ad2a95543972375d420
I20260811 22:23:04.879359 15130 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad9788d03f14ad2a95543972375d420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:04 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:46210
I20260811 22:23:04.879606 14926 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad9788d03f14ad2a95543972375d420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:04 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:41568
I20260811 22:23:04.880215 14679 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad9788d03f14ad2a95543972375d420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:04 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35812
I20260811 22:23:04.880874 15326 tablet_replica.cc:330] T 3ad9788d03f14ad2a95543972375d420 P fe0f3d599d714a39b2f7e58d23bad12f: stopping tablet replica
I20260811 22:23:04.880954 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 14568
W20260811 22:23:04.939323 14380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:41809 (error 108)
I20260811 22:23:04.940385 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 14813
I20260811 22:23:04.943454 15323 tablet_replica.cc:330] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3: stopping tablet replica
I20260811 22:23:04.943970 15323 raft_consensus.cc:2229] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:04.944433 15323 raft_consensus.cc:2258] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:04.947988 15323 ts_tablet_manager.cc:1905] T 3ad9788d03f14ad2a95543972375d420 P 1a4d71b171e34e9288826d90d31b3bf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:04.951396 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.950983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:41809 (local address 127.0.0.1:35820)
0811 22:23:04.951212 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.951298 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260811 22:23:04.951737 14380 catalog_manager.cc:4584] TS fe0f3d599d714a39b2f7e58d23bad12f (127.26.125.1:41809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ad9788d03f14ad2a95543972375d420: Network error: Client connection negotiation failed: client connection to 127.26.125.1:41809: connect: Connection refused (error 111)
I20260811 22:23:04.958642 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 15043
I20260811 22:23:04.977079 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:04.976707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:41761 (local address 127.0.0.1:41584)
0811 22:23:04.976887 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.976964 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.2:41761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 22:23:04.983436 15275 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:23:04.982746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:42365 (local address 127.0.0.1:46212)
0811 22:23:04.982962 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:04.982977 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:04.983077 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:04.983303 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.3:42365: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 22:23:04.984086 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 14354
I20260811 22:23:05.003115 27124 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:23:05.350111 27124 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:23:05.350307 27124 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:23:05.695626 27124 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:23:05.695 UTC [13661] LOG: received smart shutdown request
2026-08-11 22:23:05.700 UTC [13661] LOG: background worker "logical replication launcher" (PID 13678) exited with exit code 1
2026-08-11 22:23:05.700 UTC [13669] LOG: shutting down
2026-08-11 22:23:05.869 UTC [13661] LOG: database system is shut down
2026-08-11T22:23:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51343 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:23:06 dist-test-slave-5bn2 krb5kdc[15341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:23:06 dist-test-slave-5bn2 krb5kdc[15341](info): set up 2 sockets
Aug 11 22:23:06 dist-test-slave-5bn2 krb5kdc[15341](info): commencing operation
krb5kdc: starting...
W20260811 22:23:07.163020 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:23:07 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:23:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:23:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.125.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.125.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/rangeradmin_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/rangeradmin_127.26.125.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.125.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.125.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/rangerlookup_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/rangerlookup_127.26.125.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.125.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.125.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/HTTP_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/HTTP_127.26.125.20@KRBTEST.COM.keytab.
I20260811 22:23:07.358594 27124 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmkw4Fs/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/postgres -l logfile start
2026-08-11 22:23:09.824 UTC [15391] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:23:09.824 UTC [15391] LOG: listening on IPv4 address "127.26.125.20", port 39931
2026-08-11 22:23:09.827 UTC [15391] LOG: listening on Unix socket "/tmp/.s.PGSQL.39931"
2026-08-11 22:23:09.847 UTC [15398] LOG: database system was shut down at 2026-08-11 22:23:08 UTC
2026-08-11 22:23:09.851 UTC [15391] LOG: database system is ready to accept connections
I20260811 22:23:10.944941 27124 mini_postgres.cc:96] Postgres bound to 39931
2026-08-11 22:23:10.952 UTC [15409] FATAL: database "slave" does not exist
127.26.125.20:39931 - accepting connections
I20260811 22:23:10.952971 27124 mini_ranger.cc:161] Starting Ranger...
I20260811 22:23:10.968739 27124 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:23:11.408525 27124 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:23:11.408608 27124 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-admin
2026-08-11 22:23:11,699 [I] DB FLAVOR :POSTGRES
2026-08-11 22:23:11,699 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:23:11,699 [I] Checking connection..
2026-08-11 22:23:11,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:23:11,972 [I] Checking connection passed.
2026-08-11 22:23:11,973 [I] --------- Verifying version history table ---------
2026-08-11 22:23:11,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:23:12,232 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:23:12,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:23:12,502 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:23:12,502 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:23:12,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:23:12.767 UTC [15519] WARNING: there is no transaction in progress
2026-08-11 22:23:12,776 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:23:12,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:23:13,041 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:23:13,041 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:23:13,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:23:13,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:23:13,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5bn2.c.gcp-upstream.internal','N') ;"
2026-08-11 22:23:13,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:23:14,085 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:23:14,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:23:14,327 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:23:14,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:23:14,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:23:14,850 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:23:14,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:23:15.864 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:15.881 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:15.899 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:15.914 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:15.930 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:15.947 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.005 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.013 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.028 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.039 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.050 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.063 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.075 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.082 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.090 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16.097 UTC [15693] WARNING: there is no transaction in progress
2026-08-11 22:23:16,781 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:23:16,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:23:17,042 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:23:17,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:23:17,329 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:23:17,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:23:17,586 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:23:17,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:23:17,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:23:18,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5bn2.c.gcp-upstream.internal';"
2026-08-11 22:23:18,350 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:23:18,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:23:18,604 [I] DB_PATCHES have already been applied
I20260811 22:23:18.610737 27124 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmkw4Fs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:23:18.610816 27124 mini_ranger.cc:193] Using host: 127.26.125.20
I20260811 22:23:18.615713 27124 mini_ranger.cc:227] Ranger admin URL: http://127.26.125.20:56953
Aug 11, 2026 10:23:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:23:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:23:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:23:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:23:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:23:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:23:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:23:19.878434 27124 mini_ranger.cc:160] Time spent starting Ranger: real 8.925s user 0.004s sys 0.007s
Aug 11, 2026 10:23:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:23:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:23:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:23:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:23:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:23:42.249086 27124 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.125.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.125.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/rangerkms_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.125.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/rangerkms_127.26.125.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.125.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.125.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.125.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/HTTP_127.26.125.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.125.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/HTTP_127.26.125.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:23:42 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487022, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:23:42.350008 27124 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:23:42.361631 27124 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:23:43.184139 27124 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:23:43.184218 27124 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms
I20260811 22:23:43.184237 27124 mini_ranger_kms.cc:227] Using postgres at 127.26.125.20:39931
2026-08-11 22:23:43,354 [I] DB FLAVOR :POSTGRES
2026-08-11 22:23:43,355 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:23:43,355 [I] Checking connection
2026-08-11 22:23:43,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:23:43,610 [I] connection success
2026-08-11 22:23:43,610 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:23:43,610 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:23:43,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:23:43,877 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:23:43,878 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:23:43,878 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:23:43,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.125.20:39931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:23:44,152 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:23:44.677521 27124 mini_ranger_kms.cc:310] Created kms service
I20260811 22:23:44.906653 27124 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:23:44.951329 27124 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:23:44 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487022, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:45.388365 27124 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:23:45.388535 27124 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmkw4Fs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmkw4Fs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:23:45.388558 27124 mini_ranger_kms.cc:240] Using host: 127.26.125.20
I20260811 22:23:45.394754 27124 mini_ranger_kms.cc:276] Ranger KMS PID: 15989
I20260811 22:23:45.394838 27124 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.125.20:34413
Aug 11, 2026 10:23:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:23:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:23:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:23:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:23:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:23:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:23:47 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487027, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.125.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:48.016392 27124 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.666s user 0.010s sys 0.145s
I20260811 22:23:48.016537 27124 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:23:48.016634 27124 mini_ranger_kms.cc:401] 127.26.125.20:34413/kms/v1/keys
Aug 11 22:23:48 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487027, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.125.20@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
Aug 11 22:23:48 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487027, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.125.20@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
Aug 11 22:23:51 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:23:51.110234 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45965
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:45965
--ranger_config_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:34413/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:51.231187 16070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:51.231492 16070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:51.231532 16070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:51.235725 16070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:51.235805 16070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:51.235823 16070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:51.235838 16070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:51.235852 16070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:51.241716 16070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.125.20:34413/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:45965
--ranger_config_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:51.242976 16070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:51.244118 16070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:51.253890 16081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:51.254125 16086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:51.254258 16079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:51.254273 16070 server_base.cc:1034] running on GCE node
I20260811 22:23:51.255041 16070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:51.256372 16070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:51.257565 16070 hybrid_clock.cc:648] HybridClock initialized: now 1786487031257533 us; error 48 us; skew 500 ppm
Aug 11 22:23:51 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487031, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:51.265394 16070 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:23:51.266602 16070 webserver.cc:460] Webserver started at http://127.26.125.62:37703/ using document root <none> and password file <none>
I20260811 22:23:51.267277 16070 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:51.267343 16070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:51.267581 16070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:23:51 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487031, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:51.319278 16070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "e748b771a2954917b69afcebe759cf99"
format_stamp: "Formatted at 2026-08-11 22:23:51 on dist-test-slave-5bn2"
server_key: "07d26ca558c2c028c6ce789644cda150"
server_key_iv: "a978d8a22690b44e877f7f604be2dfb1"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:51.319996 16070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "e748b771a2954917b69afcebe759cf99"
format_stamp: "Formatted at 2026-08-11 22:23:51 on dist-test-slave-5bn2"
server_key: "07d26ca558c2c028c6ce789644cda150"
server_key_iv: "a978d8a22690b44e877f7f604be2dfb1"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:51.324494 16070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:23:51.341102 16098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:51.342597 16070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:23:51.342738 16070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "e748b771a2954917b69afcebe759cf99"
format_stamp: "Formatted at 2026-08-11 22:23:51 on dist-test-slave-5bn2"
server_key: "07d26ca558c2c028c6ce789644cda150"
server_key_iv: "a978d8a22690b44e877f7f604be2dfb1"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:51.342936 16070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:51.362900 16070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:51.366537 16070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:51.366758 16070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:51.375643 16070 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:45965
I20260811 22:23:51.375659 16179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:45965 every 8 connection(s)
I20260811 22:23:51.376719 16070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:51.379729 16180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:51.379719 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 16070
I20260811 22:23:51.379876 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:23:51.380280 27124 external_mini_cluster.cc:1428] Setting key 2df8468f72e8ea02ece452bc6ee78b7a
I20260811 22:23:51.387638 16180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Bootstrap starting.
I20260811 22:23:51.390535 16180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:51.391497 16180 log.cc:824] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:23:51 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:51.394120 16180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: No bootstrap required, opened a new log
I20260811 22:23:51.398231 16180 raft_consensus.cc:348] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } }
I20260811 22:23:51.398543 16180 raft_consensus.cc:374] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:51.398619 16180 raft_consensus.cc:729] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e748b771a2954917b69afcebe759cf99, State: Initialized, Role: FOLLOWER
I20260811 22:23:51.399184 16180 consensus_queue.cc:260] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } }
I20260811 22:23:51.399353 16180 raft_consensus.cc:388] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:51.399454 16180 raft_consensus.cc:482] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:51.399564 16180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:51.399797 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:51.381961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47140 (local address 127.26.125.62:45965)
0811 22:23:51.382463 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:51.382486 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:51.382535 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:23:51.383335 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:51.383363 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:51.383866 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:51.384285 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:51.385352 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:51.386926 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:51.388309 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:51.389026 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:51.389193 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:51.392661 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:51.392670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:51.392690 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:51.392723 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:51.395623 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:51.396250 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:51.396257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:51.396265 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:51.396418 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:51.396752 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:51.396755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:51.396757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:51.397338 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:51.397561 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:51.397889 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 22:23:51.398206 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":149,"threads_started":1}
I20260811 22:23:51.400820 16180 raft_consensus.cc:504] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } }
I20260811 22:23:51.401228 16180 leader_election.cc:302] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e748b771a2954917b69afcebe759cf99; no voters:
I20260811 22:23:51.401602 16180 leader_election.cc:288] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:23:51.401759 16189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:51.402064 16189 raft_consensus.cc:686] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 1 LEADER]: Becoming Leader. State: Replica: e748b771a2954917b69afcebe759cf99, State: Running, Role: LEADER
I20260811 22:23:51.402817 16189 consensus_queue.cc:237] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } }
I20260811 22:23:51.403214 16180 sys_catalog.cc:564] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:51.404279 16189 sys_catalog.cc:455] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e748b771a2954917b69afcebe759cf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } } }
I20260811 22:23:51.404505 16189 sys_catalog.cc:458] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:51.404778 16189 sys_catalog.cc:455] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e748b771a2954917b69afcebe759cf99. Latest consensus state: current_term: 1 leader_uuid: "e748b771a2954917b69afcebe759cf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } } }
I20260811 22:23:51.404948 16189 sys_catalog.cc:458] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:51.404965 16200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:51.405525 16180 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:23:51.410888 16200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:51.420526 16200 catalog_manager.cc:1347] Generated new cluster ID: dabd97adf6bc4d7c93ba0acbc92145ec
I20260811 22:23:51.420612 16200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:51.437095 16200 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:51.438655 16200 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:51.453405 16200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Generated new TSK 0
I20260811 22:23:51.454582 16200 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:23:52.221575 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:34413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45965
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:52.361677 16254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:52.361987 16254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:52.362033 16254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:52.366369 16254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:52.366456 16254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:52.366523 16254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:23:52.372622 16254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.125.20:34413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:52.373860 16254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:52.375008 16254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:52.387260 16261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:52.387260 16262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:52.389254 16254 server_base.cc:1034] running on GCE node
W20260811 22:23:52.389611 16264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:52.390224 16254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:52.391201 16254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:52.392405 16254 hybrid_clock.cc:648] HybridClock initialized: now 1786487032392369 us; error 47 us; skew 500 ppm
Aug 11 22:23:52 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:52.396315 16254 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:23:52.397732 16254 webserver.cc:460] Webserver started at http://127.26.125.1:34189/ using document root <none> and password file <none>
I20260811 22:23:52.398615 16254 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:52.398729 16254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:52.399035 16254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:23:52 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:52.420287 16254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "79cb4d223e6040b3b82ce25bf100d33f"
format_stamp: "Formatted at 2026-08-11 22:23:52 on dist-test-slave-5bn2"
server_key: "43e7306f586837ac1eacbae08a86523c"
server_key_iv: "0a294aaaaad6f991b035e7f92fae722e"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:52.421244 16254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "79cb4d223e6040b3b82ce25bf100d33f"
format_stamp: "Formatted at 2026-08-11 22:23:52 on dist-test-slave-5bn2"
server_key: "43e7306f586837ac1eacbae08a86523c"
server_key_iv: "0a294aaaaad6f991b035e7f92fae722e"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:52.430015 16254 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:23:52.452220 16277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:52.454270 16254 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:23:52.454442 16254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "79cb4d223e6040b3b82ce25bf100d33f"
format_stamp: "Formatted at 2026-08-11 22:23:52 on dist-test-slave-5bn2"
server_key: "43e7306f586837ac1eacbae08a86523c"
server_key_iv: "0a294aaaaad6f991b035e7f92fae722e"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:52.454604 16254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:52.483088 16254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:52.487295 16254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:52.487584 16254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:52.488487 16254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:52.489984 16254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:52.490098 16254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:52.490165 16254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:52.490233 16254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 22:23:52.507136 16254 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:46341
I20260811 22:23:52.507392 16467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:46341 every 8 connection(s)
I20260811 22:23:52.508404 16254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:52.517966 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 16254
I20260811 22:23:52.518148 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:23:52.518465 27124 external_mini_cluster.cc:1428] Setting key 69cd1a4572421d86348690caa0ac7816
Aug 11 22:23:52 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:52.532096 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:52.510741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:50191 (local address 127.26.125.62:45965)
0811 22:23:52.511089 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:52.511095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:52.512109 (+ 1014us) server_negotiation.cc:407] Connection header received
0811 22:23:52.513602 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:52.513605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:52.513659 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:52.513762 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:52.515271 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:52.516011 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:52.522715 (+ 6704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:52.523140 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:52.523259 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:52.526829 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:52.526834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:52.526837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:52.526903 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:52.528944 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:52.529776 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:52.529780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:52.529783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:52.529894 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:52.530418 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:52.530422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:52.530423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:52.530697 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:52.530778 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:52.531700 (+ 922us) server_negotiation.cc:299] Negotiation successful
0811 22:23:52.531875 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":152,"threads_started":1}
I20260811 22:23:52.534274 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:52.511233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45965 (local address 127.26.125.1:50191)
0811 22:23:52.511904 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:52.511981 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:52.513311 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:52.514098 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:52.514107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:52.514441 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:52.515082 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:52.515100 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:52.516148 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:52.516151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:52.517532 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:52.517545 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:52.523505 (+ 5960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:52.523511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:52.523773 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:52.524572 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:52.524598 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:52.526580 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:52.529159 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:52.529169 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:52.529194 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:52.529636 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:52.530034 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:52.530038 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:52.530040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:52.530290 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:52.530926 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:52.530935 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:52.531364 (+ 429us) client_negotiation.cc:769] Sending connection context
0811 22:23:52.532637 (+ 1273us) client_negotiation.cc:240] Negotiation successful
0811 22:23:52.533081 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":176,"threads_started":1}
I20260811 22:23:52.536363 16468 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45965
I20260811 22:23:52.536723 16468 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:52.537453 16468 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
I20260811 22:23:52.539633 16121 ts_manager.cc:194] Registered new tserver with Master: 79cb4d223e6040b3b82ce25bf100d33f (127.26.125.1:46341)
I20260811 22:23:52.541388 16121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:50191
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:23:52.569590 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:34413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45965
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:23:52 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:23:52.743434 16479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:52.743922 16479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:52.744072 16479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:52.750486 16479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:52.750592 16479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:52.750685 16479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:23:52.759061 16479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.125.20:34413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:52.760882 16479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:52.762676 16479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:52.777335 16493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:52.777680 16488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:52.778442 16490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:52.778638 16479 server_base.cc:1034] running on GCE node
I20260811 22:23:52.779345 16479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:52.780310 16479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:52.781570 16479 hybrid_clock.cc:648] HybridClock initialized: now 1786487032781490 us; error 99 us; skew 500 ppm
Aug 11 22:23:52 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:52.785667 16479 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:23:52.787096 16479 webserver.cc:460] Webserver started at http://127.26.125.2:44941/ using document root <none> and password file <none>
I20260811 22:23:52.787946 16479 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:52.788066 16479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:52.788421 16479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:23:52 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:52.805557 16479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "67dd97cf2d574119a4de63a5d1d3249f"
format_stamp: "Formatted at 2026-08-11 22:23:52 on dist-test-slave-5bn2"
server_key: "2186d04c261620b33a03d2ee053efb1f"
server_key_iv: "1f34bba4310919289f6c0b90ecf76ce7"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:52.807088 16479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "67dd97cf2d574119a4de63a5d1d3249f"
format_stamp: "Formatted at 2026-08-11 22:23:52 on dist-test-slave-5bn2"
server_key: "2186d04c261620b33a03d2ee053efb1f"
server_key_iv: "1f34bba4310919289f6c0b90ecf76ce7"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:52.817798 16479 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.000s
I20260811 22:23:52.839368 16504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:52.841352 16479 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:23:52.841509 16479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "67dd97cf2d574119a4de63a5d1d3249f"
format_stamp: "Formatted at 2026-08-11 22:23:52 on dist-test-slave-5bn2"
server_key: "2186d04c261620b33a03d2ee053efb1f"
server_key_iv: "1f34bba4310919289f6c0b90ecf76ce7"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:52.841660 16479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:52.857616 16479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:52.861639 16479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:52.861946 16479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:52.862838 16479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:52.863951 16479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:52.864009 16479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:52.864075 16479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:52.864106 16479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:52.875125 16479 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:44119
I20260811 22:23:52.875363 16682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:44119 every 8 connection(s)
I20260811 22:23:52.876358 16479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:52.882121 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 16479
I20260811 22:23:52.882272 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:23:52.882622 27124 external_mini_cluster.cc:1428] Setting key 0bacfa660c3c0a991029f8c42f14d135
Aug 11 22:23:52 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:52.892885 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:52.879082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:41815 (local address 127.26.125.62:45965)
0811 22:23:52.879239 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:52.879244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:52.880263 (+ 1019us) server_negotiation.cc:407] Connection header received
0811 22:23:52.881549 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:52.881553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:52.881616 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:52.881674 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:52.883477 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:52.884212 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:52.885838 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:52.886159 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:52.886262 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:52.889109 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:52.889112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:52.889114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:52.889141 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:52.890547 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:52.891169 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:52.891172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:52.891173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:52.891255 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:52.891617 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:52.891619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:52.891621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:52.891831 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:52.891920 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:52.892566 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 22:23:52.892712 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:23:52.893860 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:52.879573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45965 (local address 127.26.125.2:41815)
0811 22:23:52.880097 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:52.880156 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:52.881257 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:52.882156 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:52.882167 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:52.882540 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:52.883309 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:52.883335 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:52.884371 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:52.884374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:52.885691 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:52.885705 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:52.886519 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:52.886525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:52.886762 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:52.887444 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:52.887464 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:52.888894 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:52.890673 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:52.890680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:52.890702 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:52.891058 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:52.891342 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:52.891346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:52.891348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:52.891509 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:52.891938 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:52.891945 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:52.892237 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 22:23:52.892495 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 22:23:52.892869 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":126,"threads_started":1}
I20260811 22:23:52.895129 16684 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45965
I20260811 22:23:52.895406 16684 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:52.895956 16684 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
I20260811 22:23:52.897099 16121 ts_manager.cc:194] Registered new tserver with Master: 67dd97cf2d574119a4de63a5d1d3249f (127.26.125.2:44119)
I20260811 22:23:52.897684 16121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:41815
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:23:52.930713 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:34413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45965
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
W20260811 22:23:53.097167 16693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:53.097724 16693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:53.097860 16693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:53.104197 16693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:53.104389 16693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:53.104537 16693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:23:53.112994 16693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.125.20:34413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:53.114801 16693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:53.116487 16693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:53.130903 16699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:53.130897 16700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:53.131192 16693 server_base.cc:1034] running on GCE node
W20260811 22:23:53.131271 16702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:53.131920 16693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:53.132622 16693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:53.135871 16693 hybrid_clock.cc:648] HybridClock initialized: now 1786487033135841 us; error 51 us; skew 500 ppm
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:53.139052 16693 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:23:53.140163 16693 webserver.cc:460] Webserver started at http://127.26.125.3:39525/ using document root <none> and password file <none>
I20260811 22:23:53.140793 16693 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:53.140892 16693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:53.141155 16693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:53.169286 16693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "85aee65cfb614060a237aa4eeeb5c0af"
format_stamp: "Formatted at 2026-08-11 22:23:53 on dist-test-slave-5bn2"
server_key: "83953a2a8b8c7d8594250972324a9c9e"
server_key_iv: "eaf2407ebef10429954927009465ffc2"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:53.170159 16693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "85aee65cfb614060a237aa4eeeb5c0af"
format_stamp: "Formatted at 2026-08-11 22:23:53 on dist-test-slave-5bn2"
server_key: "83953a2a8b8c7d8594250972324a9c9e"
server_key_iv: "eaf2407ebef10429954927009465ffc2"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:53.175693 16693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 22:23:53.194878 16712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:53.196921 16693 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 22:23:53.197090 16693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "85aee65cfb614060a237aa4eeeb5c0af"
format_stamp: "Formatted at 2026-08-11 22:23:53 on dist-test-slave-5bn2"
server_key: "83953a2a8b8c7d8594250972324a9c9e"
server_key_iv: "eaf2407ebef10429954927009465ffc2"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:53.197239 16693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:53.254235 16693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:53.258158 16693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:53.258354 16693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:53.259203 16693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:53.260398 16693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:53.260475 16693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:53.260533 16693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:53.260562 16693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:53.271718 16693 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:36279
I20260811 22:23:53.271765 16904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:36279 every 8 connection(s)
I20260811 22:23:53.273053 16693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:53.278491 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 16693
I20260811 22:23:53.278633 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:23:53.279044 27124 external_mini_cluster.cc:1428] Setting key a9bf1000a1a657afbe0f23581860b6b4
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:53.299823 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:53.283029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:37665 (local address 127.26.125.62:45965)
0811 22:23:53.283294 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:53.283299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:53.283315 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:23:53.283358 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:53.283361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:53.283513 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:53.283599 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:53.286834 (+ 3235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:53.287730 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:53.289725 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:53.290112 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:53.290240 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:53.293832 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:53.293836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:53.293839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:53.293884 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:53.295990 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:53.296814 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:53.296818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:53.296820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:53.296935 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:53.298094 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:53.298098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:53.298100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:53.298367 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:53.298448 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:53.299446 (+ 998us) server_negotiation.cc:299] Negotiation successful
0811 22:23:53.299640 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260811 22:23:53.300881 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:53.275903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45965 (local address 127.26.125.3:37665)
0811 22:23:53.277225 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:53.277288 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:53.278339 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:53.285289 (+ 6950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:53.285302 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:53.285775 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:53.286610 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:53.286635 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:53.287937 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:53.287941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:53.289543 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:53.289560 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:53.290320 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:53.290324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:53.290581 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:53.291449 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:53.291475 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:53.293551 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:53.296183 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:53.296193 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:53.296218 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:53.296668 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:53.297674 (+ 1006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:53.297677 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:53.297679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:53.297954 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:53.298656 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:53.298664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:53.299115 (+ 451us) client_negotiation.cc:769] Sending connection context
0811 22:23:53.299350 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 22:23:53.299744 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"thread_start_us":179,"threads_started":1}
I20260811 22:23:53.302403 16905 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45965
I20260811 22:23:53.302719 16905 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:53.303481 16905 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
I20260811 22:23:53.304844 16121 ts_manager.cc:194] Registered new tserver with Master: 85aee65cfb614060a237aa4eeeb5c0af (127.26.125.3:36279)
I20260811 22:23:53.305696 16121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:37665
I20260811 22:23:53.310770 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:53.311697 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 16254
I20260811 22:23:53.372797 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 16479
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:53.407073 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 16693
I20260811 22:23:53.422619 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 16070
I20260811 22:23:53.444737 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45965
--webserver_interface=127.26.125.62
--webserver_port=37703
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:45965
--ranger_config_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:34413/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.125.20:34413/kms with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:53.575758 16914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:53.576097 16914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:53.576139 16914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:53.580255 16914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:53.580325 16914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:53.580344 16914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:53.580359 16914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:53.580374 16914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:53.586082 16914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.125.20:34413/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:45965
--ranger_config_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:45965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:53.587226 16914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:53.588281 16914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:53.597085 16924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:53.597102 16923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:53.597097 16927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:53.597227 16914 server_base.cc:1034] running on GCE node
I20260811 22:23:53.598064 16914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:53.599290 16914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:53.600476 16914 hybrid_clock.cc:648] HybridClock initialized: now 1786487033600440 us; error 46 us; skew 500 ppm
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:53.603410 16914 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:23:53.604394 16914 webserver.cc:460] Webserver started at http://127.26.125.62:37703/ using document root <none> and password file <none>
I20260811 22:23:53.604975 16914 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:53.605099 16914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:53.680054 16914 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:23:53.682667 16940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:53.684374 16914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:23:53.684548 16914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "e748b771a2954917b69afcebe759cf99"
format_stamp: "Formatted at 2026-08-11 22:23:51 on dist-test-slave-5bn2"
server_key: "07d26ca558c2c028c6ce789644cda150"
server_key_iv: "a978d8a22690b44e877f7f604be2dfb1"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:53.685201 16914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:53.709688 16914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:53.712641 16914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:53.712837 16914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:53.720300 17018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:45965 every 8 connection(s)
I20260811 22:23:53.720296 16914 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:45965
I20260811 22:23:53.721552 16914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:53.725697 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 16914
I20260811 22:23:53.726264 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:34413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:46341
--local_ip_for_outbound_sockets=127.26.125.1
--tserver_master_addrs=127.26.125.62:45965
--webserver_port=34189
--webserver_interface=127.26.125.1
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.125.20:34413/kms with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:23:53.726619 17019 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:23:53.728788 17019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Bootstrap starting.
I20260811 22:23:53.746002 17019 log.cc:824] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:53.753984 17019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:23:53.754421 17019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Bootstrap complete.
I20260811 22:23:53.758498 17019 raft_consensus.cc:348] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } }
I20260811 22:23:53.758936 17019 raft_consensus.cc:729] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e748b771a2954917b69afcebe759cf99, State: Initialized, Role: FOLLOWER
I20260811 22:23:53.759434 17019 consensus_queue.cc:260] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } }
I20260811 22:23:53.759549 17019 raft_consensus.cc:388] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:53.759614 17019 raft_consensus.cc:482] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:53.759706 17019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:23:53.761626 17019 raft_consensus.cc:504] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } }
I20260811 22:23:53.761971 17019 leader_election.cc:302] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e748b771a2954917b69afcebe759cf99; no voters:
I20260811 22:23:53.762317 17019 leader_election.cc:288] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:23:53.762413 17024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:23:53.762629 17024 raft_consensus.cc:686] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [term 2 LEADER]: Becoming Leader. State: Replica: e748b771a2954917b69afcebe759cf99, State: Running, Role: LEADER
I20260811 22:23:53.763032 17024 consensus_queue.cc:237] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } }
I20260811 22:23:53.763648 17019 sys_catalog.cc:564] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:53.764458 17025 sys_catalog.cc:455] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e748b771a2954917b69afcebe759cf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } } }
I20260811 22:23:53.764595 17025 sys_catalog.cc:458] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:53.764977 17027 sys_catalog.cc:455] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e748b771a2954917b69afcebe759cf99. Latest consensus state: current_term: 2 leader_uuid: "e748b771a2954917b69afcebe759cf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e748b771a2954917b69afcebe759cf99" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 45965 } } }
I20260811 22:23:53.765121 17027 sys_catalog.cc:458] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:53.765823 17040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:53.766007 17019 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:23:53.772286 17040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:53.773175 17040 catalog_manager.cc:1259] Loaded cluster ID: dabd97adf6bc4d7c93ba0acbc92145ec
I20260811 22:23:53.773290 17040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:53.776043 17040 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:53.776947 17040 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Loaded TSK: 0
I20260811 22:23:53.778437 17040 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:23:53.877270 17021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:53.877856 17021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:53.877998 17021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:53.883795 17021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:53.883971 17021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:53.884114 17021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:23:53.892369 17021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.125.20:34413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:46341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=34189
--tserver_master_addrs=127.26.125.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:53.893886 17021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:53.895512 17021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:53.909296 17085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:53.909725 17080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:53.909741 17082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:53.911587 17021 server_base.cc:1034] running on GCE node
I20260811 22:23:53.912572 17021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:53.914019 17021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:53.915220 17021 hybrid_clock.cc:648] HybridClock initialized: now 1786487033915142 us; error 95 us; skew 500 ppm
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:53.920004 17021 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:23:53.921451 17021 webserver.cc:460] Webserver started at http://127.26.125.1:34189/ using document root <none> and password file <none>
I20260811 22:23:53.922283 17021 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:53.922452 17021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:23:53 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:53.958645 17021 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:23:53.979718 17102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:53.981796 17021 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.002s sys 0.000s
I20260811 22:23:53.981976 17021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "79cb4d223e6040b3b82ce25bf100d33f"
format_stamp: "Formatted at 2026-08-11 22:23:52 on dist-test-slave-5bn2"
server_key: "43e7306f586837ac1eacbae08a86523c"
server_key_iv: "0a294aaaaad6f991b035e7f92fae722e"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:53.982749 17021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:54.003688 17021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:54.007985 17021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:54.008389 17021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:54.009325 17021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:54.010643 17021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:54.010753 17021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 22:23:54.010876 17021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:54.010957 17021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.022105 17021 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:46341
I20260811 22:23:54.024154 17269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:46341 every 8 connection(s)
I20260811 22:23:54.025290 17021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:54.029500 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 17021
I20260811 22:23:54.030016 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:34413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:44119
--local_ip_for_outbound_sockets=127.26.125.2
--tserver_master_addrs=127.26.125.62:45965
--webserver_port=44941
--webserver_interface=127.26.125.2
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.125.20:34413/kms with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:23:54 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:54.052366 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.028470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:46863 (local address 127.26.125.62:45965)
0811 22:23:54.029609 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:54.029634 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:54.031441 (+ 1807us) server_negotiation.cc:407] Connection header received
0811 22:23:54.033074 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:54.033094 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:54.033602 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:54.034059 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:54.036158 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.037882 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.040132 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.041012 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.041182 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.045198 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:54.045207 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:54.045229 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:54.045272 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:54.047842 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.048667 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.048675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.048682 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.048866 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.049369 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.049373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.049375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.049850 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:54.049957 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:54.050743 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 22:23:54.051146 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":143,"threads_started":1}
I20260811 22:23:54.056046 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.029630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45965 (local address 127.26.125.1:46863)
0811 22:23:54.031202 (+ 1572us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:54.031271 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:54.032433 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:54.034681 (+ 2248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:54.034692 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:54.035147 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:54.035894 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.035919 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.038124 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.038128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.039936 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.039953 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.041532 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.041537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.041769 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.042668 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:54.042695 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:54.044872 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:54.048031 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.048039 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.048060 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.048488 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.049017 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.049020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.049023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.049225 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.050024 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:54.050030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:54.050312 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 22:23:54.050919 (+ 607us) client_negotiation.cc:240] Negotiation successful
0811 22:23:54.055245 (+ 4326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1313,"thread_start_us":158,"threads_started":1}
I20260811 22:23:54.059260 17271 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45965
I20260811 22:23:54.059553 17271 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.060215 17271 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
W20260811 22:23:54.062912 17271 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:45965 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:23:54.063062 17271 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.063279 17271 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
I20260811 22:23:54.064244 17271 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.064455 17271 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
W20260811 22:23:54.065253 17271 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:23:54.205695 17276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:54.206019 17276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:54.206081 17276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:54.210412 17276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:54.210489 17276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:54.210574 17276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:23:54.218436 17276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.125.20:34413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:44119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=44941
--tserver_master_addrs=127.26.125.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:54.220046 17276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:54.221421 17276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:54.235550 17284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:54.238397 17285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:54.238636 17289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:54.239673 17276 server_base.cc:1034] running on GCE node
I20260811 22:23:54.240221 17276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:54.240958 17276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:54.242151 17276 hybrid_clock.cc:648] HybridClock initialized: now 1786487034242088 us; error 76 us; skew 500 ppm
Aug 11 22:23:54 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:54.246060 17276 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:23:54.247134 17276 webserver.cc:460] Webserver started at http://127.26.125.2:44941/ using document root <none> and password file <none>
I20260811 22:23:54.247752 17276 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:54.247843 17276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:23:54 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:54.278564 17276 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:23:54.283648 17300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.285672 17276 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 22:23:54.285840 17276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "67dd97cf2d574119a4de63a5d1d3249f"
format_stamp: "Formatted at 2026-08-11 22:23:52 on dist-test-slave-5bn2"
server_key: "2186d04c261620b33a03d2ee053efb1f"
server_key_iv: "1f34bba4310919289f6c0b90ecf76ce7"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:54.286667 17276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:54.308203 17276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:54.312325 17276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:54.312637 17276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:54.313532 17276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:54.314707 17276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:54.314831 17276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.314981 17276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:54.315055 17276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.328869 17276 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:44119
I20260811 22:23:54.328900 17474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:44119 every 8 connection(s)
I20260811 22:23:54.330346 17276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:54.336359 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 17276
I20260811 22:23:54.336867 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.125.20:34413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:36279
--local_ip_for_outbound_sockets=127.26.125.3
--tserver_master_addrs=127.26.125.62:45965
--webserver_port=39525
--webserver_interface=127.26.125.3
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.125.20:34413/kms with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:23:54 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:54.352835 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.332427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:35705 (local address 127.26.125.62:45965)
0811 22:23:54.332554 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:54.332558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:54.333719 (+ 1161us) server_negotiation.cc:407] Connection header received
0811 22:23:54.334959 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:54.334963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:54.335028 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:54.335131 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:54.336382 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.337145 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.338738 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.339100 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.339186 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.347715 (+ 8529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:54.347720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:54.347723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:54.347762 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:54.349695 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.350558 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.350563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.350565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.350669 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.351227 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.351231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.351233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.351519 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:54.351634 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:54.352492 (+ 858us) server_negotiation.cc:299] Negotiation successful
0811 22:23:54.352654 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:23:54.354072 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.333015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45965 (local address 127.26.125.2:35705)
0811 22:23:54.333560 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:54.333613 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:54.334672 (+ 1059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:54.335311 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:54.335319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:54.335616 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:54.336195 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.336216 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.337308 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.337310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.338584 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.338597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.339420 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.339424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.339669 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.340395 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:54.340418 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:54.347400 (+ 6982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:54.349886 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.349895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.349922 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.350421 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.350795 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.350799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.350801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.351092 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.351674 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:54.351683 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:54.352133 (+ 450us) client_negotiation.cc:769] Sending connection context
0811 22:23:54.352383 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 22:23:54.352805 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":165,"threads_started":1}
I20260811 22:23:54.355752 17475 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45965
I20260811 22:23:54.356117 17475 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.356766 17475 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
W20260811 22:23:54.358807 17475 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:45965 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:23:54.358997 17475 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.359205 17475 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
I20260811 22:23:54.360236 17475 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.360430 17475 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
W20260811 22:23:54.361331 17475 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:23:54.488456 17482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:54.488804 17482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:54.488859 17482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:54.493571 17482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:54.493659 17482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:54.493729 17482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:23:54.499950 17482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.125.20:34413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:36279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=39525
--tserver_master_addrs=127.26.125.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:54.501190 17482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:54.502388 17482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:54.515103 17487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:54.516676 17491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:54.517237 17482 server_base.cc:1034] running on GCE node
W20260811 22:23:54.517439 17488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:54.518831 17482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:54.519848 17482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:54.521020 17482 hybrid_clock.cc:648] HybridClock initialized: now 1786487034520976 us; error 65 us; skew 500 ppm
Aug 11 22:23:54 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:54.524557 17482 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:23:54.525717 17482 webserver.cc:460] Webserver started at http://127.26.125.3:39525/ using document root <none> and password file <none>
I20260811 22:23:54.526515 17482 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:54.526625 17482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:23:54 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:54.558936 17482 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.001s
I20260811 22:23:54.562573 17503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.564399 17482 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 22:23:54.564563 17482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "85aee65cfb614060a237aa4eeeb5c0af"
format_stamp: "Formatted at 2026-08-11 22:23:53 on dist-test-slave-5bn2"
server_key: "83953a2a8b8c7d8594250972324a9c9e"
server_key_iv: "eaf2407ebef10429954927009465ffc2"
server_key_version: "kuduclusterkey@0"
I20260811 22:23:54.565378 17482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:54.585299 17482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:54.589578 17482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:54.589823 17482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:54.590521 17482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:54.591601 17482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:54.591657 17482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.591714 17482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:54.591739 17482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.602722 17482 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:36279
I20260811 22:23:54.603963 17482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:54.605741 17684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:36279 every 8 connection(s)
I20260811 22:23:54.608546 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 17482
I20260811 22:23:54.631662 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.609633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.26.125.62:45965)
0811 22:23:54.615969 (+ 6336us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:54.615973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:54.615984 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:23:54.616035 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:54.616039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:54.616097 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:54.616197 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:54.617075 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.617614 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.619004 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.619811 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.619924 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.621887 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:54.621889 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:54.621891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:54.621915 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:54.628461 (+ 6546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.629151 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.629154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.629156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.629264 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.630023 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.630025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.630026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.630208 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:54.630418 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:54.631350 (+ 932us) server_negotiation.cc:299] Negotiation successful
0811 22:23:54.631500 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6244,"thread_start_us":88,"threads_started":1}
Aug 11 22:23:54 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:54.637267 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.607843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:59567 (local address 127.26.125.62:45965)
0811 22:23:54.608014 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:54.608020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:54.615783 (+ 7763us) server_negotiation.cc:407] Connection header received
0811 22:23:54.617682 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:54.617685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:54.617739 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:54.617809 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:54.621070 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.621786 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.629610 (+ 7824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.629869 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.629929 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.633642 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:54.633645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:54.633647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:54.633671 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:54.635026 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.635594 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.635597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.635599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.635677 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.636067 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.636071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.636072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.636269 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:54.636357 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:54.636957 (+ 600us) server_negotiation.cc:299] Negotiation successful
0811 22:23:54.637096 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:23:54.638410 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.608379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:45965 (local address 127.26.125.3:59567)
0811 22:23:54.615559 (+ 7180us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:54.615654 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:54.617417 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:54.618105 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:54.618117 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:54.618535 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:54.620870 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.620899 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.622272 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.622276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.629456 (+ 7180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.629473 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.630616 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.630621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.630906 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.631966 (+ 1060us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:54.631989 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:54.633407 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:54.635149 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.635156 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.635179 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.635477 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.635767 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.635771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.635773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.635960 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.636377 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:54.636386 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:54.636691 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 22:23:54.636879 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 22:23:54.637309 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6845,"thread_start_us":183,"threads_started":1}
I20260811 22:23:54.639782 17686 heartbeater.cc:347] Connected to a master server at 127.26.125.62:45965
I20260811 22:23:54.640081 17686 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.640723 17686 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
I20260811 22:23:54.642184 16966 ts_manager.cc:194] Registered new tserver with Master: 85aee65cfb614060a237aa4eeeb5c0af (127.26.125.3:36279)
I20260811 22:23:54.643749 16966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:59567
Aug 11 22:23:54 dist-test-slave-5bn2 krb5kdc[15341](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:55.065505 17271 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:55.065884 17271 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
I20260811 22:23:55.066833 16966 ts_manager.cc:194] Registered new tserver with Master: 79cb4d223e6040b3b82ce25bf100d33f (127.26.125.1:46341)
I20260811 22:23:55.067694 16966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:46863
I20260811 22:23:55.361603 17475 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:55.361924 17475 heartbeater.cc:511] Master 127.26.125.62:45965 requested a full tablet report, sending...
I20260811 22:23:55.363003 16966 ts_manager.cc:194] Registered new tserver with Master: 67dd97cf2d574119a4de63a5d1d3249f (127.26.125.2:44119)
I20260811 22:23:55.363831 16966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:35705
Aug 11 22:23:55 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:55.367624 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:55.383008 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.369299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47166 (local address 127.26.125.62:45965)
0811 22:23:55.369574 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.369579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.369643 (+ 64us) server_negotiation.cc:407] Connection header received
0811 22:23:55.369787 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.369790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.369845 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.369947 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:55.374697 (+ 4750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.375549 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.376811 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.377172 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.377284 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.379184 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:55.379189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:55.379192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:55.379227 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:55.381059 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:55.381611 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:55.381615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:55.381617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:55.381730 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:55.382017 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:55.382020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:55.382022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:55.382247 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:55.382377 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.382638 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.382783 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":102,"threads_started":1}
I20260811 22:23:55.387989 16966 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:23:55.393574 16966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:23:55.432340 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.421415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46341 (local address 127.0.0.1:50390)
0811 22:23:55.424220 (+ 2805us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.424230 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.424295 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.426009 (+ 1714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.426038 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.426067 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.426221 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.426228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.427837 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.427841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.430250 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.430264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.431716 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.431721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.431851 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.431896 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.432029 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.432169 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2758,"thread_start_us":93,"threads_started":1}
I20260811 22:23:55.432807 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.421817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.26.125.1:46341)
0811 22:23:55.423274 (+ 1457us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.423280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.424382 (+ 1102us) server_negotiation.cc:407] Connection header received
0811 22:23:55.424475 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.424480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.424719 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.424861 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.426350 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.427705 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.430422 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.431354 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.431466 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.431639 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.432551 (+ 912us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.432666 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1364,"thread_start_us":85,"threads_started":1}
I20260811 22:23:55.436385 17149 tablet_service.cc:1467] Processing CreateTablet for tablet 9ddc9b3b86bf442891d2c58e2a116c79 (DEFAULT_TABLE table=test-table [id=2952a5126bfd44d18da3dae609712a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:55.437763 17149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ddc9b3b86bf442891d2c58e2a116c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:55.457732 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.421050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:44119 (local address 127.0.0.1:46014)
0811 22:23:55.423986 (+ 2936us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.423998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.424075 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.431967 (+ 7892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.431982 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.432001 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.432086 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.432092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.449007 (+ 16915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.449010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.451388 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.451401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.456151 (+ 4750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.456155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.456280 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.456312 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.456392 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.456481 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2865,"thread_start_us":103,"threads_started":1}
I20260811 22:23:55.458029 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.417958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:36279 (local address 127.0.0.1:59004)
0811 22:23:55.423547 (+ 5589us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.423611 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.423757 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.431151 (+ 7394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.431154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.431173 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.431272 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.431277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.446161 (+ 14884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.446166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.447449 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.447464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.457575 (+ 10111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.457580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.457688 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.457719 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.457775 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.457860 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5383,"thread_start_us":126,"threads_started":1}
I20260811 22:23:55.458751 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.419293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59004 (local address 127.26.125.3:36279)
0811 22:23:55.423167 (+ 3874us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.423175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.430438 (+ 7263us) server_negotiation.cc:407] Connection header received
0811 22:23:55.430542 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.430549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.430805 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.430994 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.444751 (+ 13757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.445999 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.456439 (+ 10440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.457394 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.458254 (+ 860us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.458439 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.458502 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.458607 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3742,"thread_start_us":90,"threads_started":1}
I20260811 22:23:55.459347 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.421239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.26.125.2:44119)
0811 22:23:55.423344 (+ 2105us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.423350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.431397 (+ 8047us) server_negotiation.cc:407] Connection header received
0811 22:23:55.431485 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.431491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.431690 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.431825 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.447632 (+ 15807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.448847 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.455027 (+ 6180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.455991 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.456134 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.456313 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.459049 (+ 2736us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.459201 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2009,"thread_start_us":107,"threads_started":1}
I20260811 22:23:55.460623 17737 tablet_bootstrap.cc:492] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f: Bootstrap starting.
I20260811 22:23:55.462149 17372 tablet_service.cc:1467] Processing CreateTablet for tablet 9ddc9b3b86bf442891d2c58e2a116c79 (DEFAULT_TABLE table=test-table [id=2952a5126bfd44d18da3dae609712a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:55.462152 17551 tablet_service.cc:1467] Processing CreateTablet for tablet 9ddc9b3b86bf442891d2c58e2a116c79 (DEFAULT_TABLE table=test-table [id=2952a5126bfd44d18da3dae609712a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:55.463335 17737 tablet_bootstrap.cc:654] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:55.463665 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ddc9b3b86bf442891d2c58e2a116c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:55.463690 17551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ddc9b3b86bf442891d2c58e2a116c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:55.464295 17737 log.cc:824] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:55.473944 17741 tablet_bootstrap.cc:492] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Bootstrap starting.
I20260811 22:23:55.476562 17741 tablet_bootstrap.cc:654] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:55.477372 17743 tablet_bootstrap.cc:492] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Bootstrap starting.
I20260811 22:23:55.477560 17741 log.cc:824] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:55.480310 17741 tablet_bootstrap.cc:492] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: No bootstrap required, opened a new log
I20260811 22:23:55.480593 17741 ts_tablet_manager.cc:1397] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 22:23:55.480971 17737 tablet_bootstrap.cc:492] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f: No bootstrap required, opened a new log
I20260811 22:23:55.481175 17737 ts_tablet_manager.cc:1397] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260811 22:23:55.485049 17741 raft_consensus.cc:348] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.485466 17741 raft_consensus.cc:374] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:55.485522 17741 raft_consensus.cc:729] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67dd97cf2d574119a4de63a5d1d3249f, State: Initialized, Role: FOLLOWER
I20260811 22:23:55.485361 17737 raft_consensus.cc:348] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.485955 17737 raft_consensus.cc:374] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:55.486054 17737 raft_consensus.cc:729] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79cb4d223e6040b3b82ce25bf100d33f, State: Initialized, Role: FOLLOWER
I20260811 22:23:55.486094 17741 consensus_queue.cc:260] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.486716 17737 consensus_queue.cc:260] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.487452 17741 ts_tablet_manager.cc:1428] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:23:55.487910 17475 heartbeater.cc:503] Master 127.26.125.62:45965 was elected leader, sending a full tablet report...
I20260811 22:23:55.487953 17737 ts_tablet_manager.cc:1428] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:23:55.488199 17271 heartbeater.cc:503] Master 127.26.125.62:45965 was elected leader, sending a full tablet report...
I20260811 22:23:55.491727 17743 tablet_bootstrap.cc:654] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:55.493057 17743 log.cc:824] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:55.495874 17743 tablet_bootstrap.cc:492] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: No bootstrap required, opened a new log
I20260811 22:23:55.496130 17743 ts_tablet_manager.cc:1397] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260811 22:23:55.500067 17743 raft_consensus.cc:348] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.500360 17743 raft_consensus.cc:374] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:55.500442 17743 raft_consensus.cc:729] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85aee65cfb614060a237aa4eeeb5c0af, State: Initialized, Role: FOLLOWER
I20260811 22:23:55.500967 17743 consensus_queue.cc:260] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.501740 17686 heartbeater.cc:503] Master 127.26.125.62:45965 was elected leader, sending a full tablet report...
I20260811 22:23:55.502228 17743 ts_tablet_manager.cc:1428] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 22:23:55.527585 17272 tablet.cc:2369] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:55.582239 17479 tablet.cc:2369] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:55.607059 17688 tablet.cc:2369] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:23:55 dist-test-slave-5bn2 krb5kdc[15341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for HTTP/127.26.125.20@KRBTEST.COM
I20260811 22:23:55.705812 17745 raft_consensus.cc:482] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:55.706094 17745 raft_consensus.cc:504] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.707630 17745 leader_election.cc:288] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85aee65cfb614060a237aa4eeeb5c0af (127.26.125.3:36279), 79cb4d223e6040b3b82ce25bf100d33f (127.26.125.1:46341)
I20260811 22:23:55.714972 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.709029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:39621 (local address 127.26.125.3:36279)
0811 22:23:55.709202 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.709208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.709496 (+ 288us) server_negotiation.cc:407] Connection header received
0811 22:23:55.709716 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.709720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.709771 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.709821 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.710332 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.711175 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.713176 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.713808 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.713910 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.713969 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.714703 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.714796 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:23:55.715466 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.708080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:36279 (local address 127.26.125.2:39621)
0811 22:23:55.709395 (+ 1315us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.709411 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.709587 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.709977 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.709980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.710003 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.710177 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.710184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.711352 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.711355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.713006 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.713020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.714077 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.714080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.714185 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.714211 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.715213 (+ 1002us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.715302 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1222,"thread_start_us":145,"threads_started":1}
I20260811 22:23:55.716203 17586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ddc9b3b86bf442891d2c58e2a116c79" candidate_uuid: "67dd97cf2d574119a4de63a5d1d3249f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85aee65cfb614060a237aa4eeeb5c0af" is_pre_election: true
I20260811 22:23:55.716614 17586 raft_consensus.cc:2454] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67dd97cf2d574119a4de63a5d1d3249f in term 0.
I20260811 22:23:55.717195 17303 leader_election.cc:302] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67dd97cf2d574119a4de63a5d1d3249f, 85aee65cfb614060a237aa4eeeb5c0af; no voters:
I20260811 22:23:55.717561 17745 raft_consensus.cc:2781] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:55.717662 17745 raft_consensus.cc:482] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:55.717715 17745 raft_consensus.cc:3037] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:55.719106 17745 raft_consensus.cc:504] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.719530 17745 leader_election.cc:288] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [CANDIDATE]: Term 1 election: Requested vote from peers 85aee65cfb614060a237aa4eeeb5c0af (127.26.125.3:36279), 79cb4d223e6040b3b82ce25bf100d33f (127.26.125.1:46341)
I20260811 22:23:55.720059 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.709173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:38749 (local address 127.26.125.1:46341)
0811 22:23:55.709326 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.709330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.714612 (+ 5282us) server_negotiation.cc:407] Connection header received
0811 22:23:55.716093 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.716097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.716152 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.716196 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.716687 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.717365 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.718754 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.719320 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.719432 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.719489 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.719843 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.719933 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:23:55.720481 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.708531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46341 (local address 127.26.125.2:38749)
0811 22:23:55.714516 (+ 5985us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.714532 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.715944 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.716340 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.716343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.716363 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.716486 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.716492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.717511 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.717514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.718610 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.718621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.719599 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.719602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.719699 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.719723 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.720254 (+ 531us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.720340 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5881,"thread_start_us":94,"threads_started":1}
I20260811 22:23:55.721622 17748 raft_consensus.cc:482] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:55.721812 17748 raft_consensus.cc:504] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.723097 17748 leader_election.cc:288] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85aee65cfb614060a237aa4eeeb5c0af (127.26.125.3:36279), 67dd97cf2d574119a4de63a5d1d3249f (127.26.125.2:44119)
I20260811 22:23:55.728243 17189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ddc9b3b86bf442891d2c58e2a116c79" candidate_uuid: "67dd97cf2d574119a4de63a5d1d3249f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79cb4d223e6040b3b82ce25bf100d33f"
I20260811 22:23:55.728514 17189 raft_consensus.cc:3037] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:55.729969 17189 raft_consensus.cc:2454] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67dd97cf2d574119a4de63a5d1d3249f in term 1.
I20260811 22:23:55.730489 17303 leader_election.cc:302] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67dd97cf2d574119a4de63a5d1d3249f, 79cb4d223e6040b3b82ce25bf100d33f; no voters:
I20260811 22:23:55.731298 17190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ddc9b3b86bf442891d2c58e2a116c79" candidate_uuid: "67dd97cf2d574119a4de63a5d1d3249f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79cb4d223e6040b3b82ce25bf100d33f" is_pre_election: true
I20260811 22:23:55.731473 17190 raft_consensus.cc:2362] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 67dd97cf2d574119a4de63a5d1d3249f in term 1. Re-sending same reply.
I20260811 22:23:55.732203 17586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ddc9b3b86bf442891d2c58e2a116c79" candidate_uuid: "67dd97cf2d574119a4de63a5d1d3249f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85aee65cfb614060a237aa4eeeb5c0af"
I20260811 22:23:55.732357 17586 raft_consensus.cc:3037] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:55.733628 17586 raft_consensus.cc:2454] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67dd97cf2d574119a4de63a5d1d3249f in term 1.
I20260811 22:23:55.734112 17745 raft_consensus.cc:2781] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:55.734423 17745 raft_consensus.cc:686] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 1 LEADER]: Becoming Leader. State: Replica: 67dd97cf2d574119a4de63a5d1d3249f, State: Running, Role: LEADER
I20260811 22:23:55.734822 17745 consensus_queue.cc:237] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } }
I20260811 22:23:55.740247 16966 catalog_manager.cc:5495] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f reported cstate change: term changed from 0 to 1, leader changed from <none> to 67dd97cf2d574119a4de63a5d1d3249f (127.26.125.2). New cstate: current_term: 1 leader_uuid: "67dd97cf2d574119a4de63a5d1d3249f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67dd97cf2d574119a4de63a5d1d3249f" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 44119 } health_report { overall_health: HEALTHY } } }
I20260811 22:23:55.743834 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.727941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:37019 (local address 127.26.125.2:44119)
0811 22:23:55.728095 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.728101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.739765 (+ 11664us) server_negotiation.cc:407] Connection header received
0811 22:23:55.739882 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.739885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.739934 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.740014 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.740412 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.741092 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.742489 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.743125 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.743216 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.743271 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.743618 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.743704 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:23:55.744239 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.727290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:44119 (local address 127.26.125.1:37019)
0811 22:23:55.739678 (+ 12388us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.739693 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.739800 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.740138 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.740141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.740160 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.740281 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.740288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.741232 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.741235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.742336 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.742349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.743363 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.743367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.743471 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.743496 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.743994 (+ 498us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.744082 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12297,"thread_start_us":107,"threads_started":1}
I20260811 22:23:55.744870 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ddc9b3b86bf442891d2c58e2a116c79" candidate_uuid: "79cb4d223e6040b3b82ce25bf100d33f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67dd97cf2d574119a4de63a5d1d3249f" is_pre_election: true
I20260811 22:23:55.748562 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.727773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:38101 (local address 127.26.125.3:36279)
0811 22:23:55.728097 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.728102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.731798 (+ 3696us) server_negotiation.cc:407] Connection header received
0811 22:23:55.731993 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.731997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.732057 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.732103 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.733016 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.733801 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.743211 (+ 9410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.743788 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.743891 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.743942 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.748338 (+ 4396us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.748439 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260811 22:23:55.748597 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.726828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:36279 (local address 127.26.125.1:38101)
0811 22:23:55.731685 (+ 4857us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.731702 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.731880 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.732238 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.732240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.732256 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.732524 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.732531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.734001 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.734004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.737081 (+ 3077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.737095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.744222 (+ 7127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.744225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.748136 (+ 3911us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.748181 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.748298 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.748408 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4767,"thread_start_us":122,"threads_started":1}
I20260811 22:23:55.749073 17586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ddc9b3b86bf442891d2c58e2a116c79" candidate_uuid: "79cb4d223e6040b3b82ce25bf100d33f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85aee65cfb614060a237aa4eeeb5c0af" is_pre_election: true
I20260811 22:23:55.749295 17586 raft_consensus.cc:2379] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79cb4d223e6040b3b82ce25bf100d33f in current term 1: Already voted for candidate 67dd97cf2d574119a4de63a5d1d3249f in this term.
I20260811 22:23:55.749878 17104 leader_election.cc:302] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79cb4d223e6040b3b82ce25bf100d33f; no voters: 67dd97cf2d574119a4de63a5d1d3249f, 85aee65cfb614060a237aa4eeeb5c0af
I20260811 22:23:55.750528 17748 raft_consensus.cc:2726] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 22:23:55.812245 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.808030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.26.125.2:44119)
0811 22:23:55.808169 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.808173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.808186 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:23:55.808285 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.808289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.808358 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.808474 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:23:55.808734 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.809486 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.810673 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.810938 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.811033 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.811348 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:23:55.811811 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:23:55.811926 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.812007 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.812074 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:23:55.822190 17586 raft_consensus.cc:1261] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 1 FOLLOWER]: Refusing update from remote peer 67dd97cf2d574119a4de63a5d1d3249f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:55.822237 17190 raft_consensus.cc:1261] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 1 FOLLOWER]: Refusing update from remote peer 67dd97cf2d574119a4de63a5d1d3249f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:55.823155 17760 consensus_queue.cc:1035] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [LEADER]: Connected to new peer: Peer: permanent_uuid: "85aee65cfb614060a237aa4eeeb5c0af" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 36279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:55.823348 17745 consensus_queue.cc:1035] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79cb4d223e6040b3b82ce25bf100d33f" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:55.834396 17766 mvcc.cc:205] Tried to move back new op lower bound from 7317450898717663232 to 7317450898380767232. Current Snapshot: MvccSnapshot[applied={T|T < 7317450898717663232}]
I20260811 22:23:55.844312 17769 mvcc.cc:205] Tried to move back new op lower bound from 7317450898717663232 to 7317450898380767232. Current Snapshot: MvccSnapshot[applied={T|T < 7317450898717663232}]
I20260811 22:23:55.852927 16966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:55.853154 16966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:55.856287 16966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e748b771a2954917b69afcebe759cf99: Sending DeleteTablet for 3 replicas of tablet 9ddc9b3b86bf442891d2c58e2a116c79
I20260811 22:23:55.857605 17149 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ddc9b3b86bf442891d2c58e2a116c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:55 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:50390
I20260811 22:23:55.858223 17781 tablet_replica.cc:330] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f: stopping tablet replica
I20260811 22:23:55.858614 17781 raft_consensus.cc:2229] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:55.858666 17551 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ddc9b3b86bf442891d2c58e2a116c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:55 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:59004
I20260811 22:23:55.859056 17781 raft_consensus.cc:2258] T 9ddc9b3b86bf442891d2c58e2a116c79 P 79cb4d223e6040b3b82ce25bf100d33f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:55.859588 17372 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ddc9b3b86bf442891d2c58e2a116c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:55 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:46014
I20260811 22:23:55.860436 17783 tablet_replica.cc:330] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: stopping tablet replica
I20260811 22:23:55.860855 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 17021
I20260811 22:23:55.860764 17783 raft_consensus.cc:2229] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:55.888633 17783 raft_consensus.cc:2258] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:55.890705 17783 ts_tablet_manager.cc:1905] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:55.891134 17784 tablet_replica.cc:330] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: stopping tablet replica
I20260811 22:23:55.891616 17784 raft_consensus.cc:2229] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:23:55.893247 17784 raft_consensus.cc:2258] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:55.893996 17783 ts_tablet_manager.cc:1918] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:55.894126 17783 log.cc:1196] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/9ddc9b3b86bf442891d2c58e2a116c79
I20260811 22:23:55.894500 17783 ts_tablet_manager.cc:1939] T 9ddc9b3b86bf442891d2c58e2a116c79 P 85aee65cfb614060a237aa4eeeb5c0af: Deleting consensus metadata
I20260811 22:23:55.896016 16944 catalog_manager.cc:4841] TS 85aee65cfb614060a237aa4eeeb5c0af (127.26.125.3:36279): tablet 9ddc9b3b86bf442891d2c58e2a116c79 (table test-table [id=2952a5126bfd44d18da3dae609712a0a]) successfully deleted
I20260811 22:23:55.896661 17784 ts_tablet_manager.cc:1905] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:55.899852 17784 ts_tablet_manager.cc:1918] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:55.899987 17784 log.cc:1196] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/9ddc9b3b86bf442891d2c58e2a116c79
I20260811 22:23:55.900303 17784 ts_tablet_manager.cc:1939] T 9ddc9b3b86bf442891d2c58e2a116c79 P 67dd97cf2d574119a4de63a5d1d3249f: Deleting consensus metadata
W20260811 22:23:55.901809 16944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:46341 (error 108)
I20260811 22:23:55.901845 16945 catalog_manager.cc:4841] TS 67dd97cf2d574119a4de63a5d1d3249f (127.26.125.2:44119): tablet 9ddc9b3b86bf442891d2c58e2a116c79 (table test-table [id=2952a5126bfd44d18da3dae609712a0a]) successfully deleted
I20260811 22:23:55.903090 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 17276
I20260811 22:23:55.903513 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.903105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46341 (local address 127.0.0.1:50392)
0811 22:23:55.903324 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.903416 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260811 22:23:55.913569 16944 catalog_manager.cc:4584] TS 79cb4d223e6040b3b82ce25bf100d33f (127.26.125.1:46341): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ddc9b3b86bf442891d2c58e2a116c79: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46341: connect: Connection refused (error 111)
I20260811 22:23:55.919754 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 17482
I20260811 22:23:55.935608 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 16914
I20260811 22:23:55.952140 27124 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:23:56.298713 27124 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:23:56.298866 27124 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:23:56.644490 27124 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:23:56.644 UTC [15391] LOG: received smart shutdown request
2026-08-11 22:23:56.649 UTC [15391] LOG: background worker "logical replication launcher" (PID 15407) exited with exit code 1
2026-08-11 22:23:56.649 UTC [15399] LOG: shutting down
2026-08-11 22:23:56.806 UTC [15391] LOG: database system is shut down
2026-08-11T22:23:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50941 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:23:56 dist-test-slave-5bn2 krb5kdc[17800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:23:56 dist-test-slave-5bn2 krb5kdc[17800](info): set up 2 sockets
Aug 11 22:23:56 dist-test-slave-5bn2 krb5kdc[17800](info): commencing operation
krb5kdc: starting...
W20260811 22:23:58.102134 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:23:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:23:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:23:58.246088 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36419
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36419,127.26.125.61:43297,127.26.125.60:41243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:58.365773 17816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:58.366096 17816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:58.366139 17816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:58.370395 17816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:58.370479 17816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:58.370498 17816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:58.370513 17816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:58.370527 17816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:58.376153 17816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36419,127.26.125.61:43297,127.26.125.60:41243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:58.377300 17816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:58.378355 17816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:58.387378 17828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.387368 17823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.387368 17824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:58.387602 17816 server_base.cc:1034] running on GCE node
I20260811 22:23:58.388589 17816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:58.389904 17816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:58.391109 17816 hybrid_clock.cc:648] HybridClock initialized: now 1786487038391093 us; error 41 us; skew 500 ppm
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:58.394153 17816 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:23:58.395196 17816 webserver.cc:460] Webserver started at http://127.26.125.62:33825/ using document root <none> and password file <none>
I20260811 22:23:58.395771 17816 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:58.395833 17816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:58.396005 17816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:58.398173 17816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "f4f8aa157afa427ab80f473e879b703a"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "1548b98949ed5cacd4287dc0b2db2574"
server_key_iv: "fd5274e10c9b0067e204a33c99d57180"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.398744 17816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "f4f8aa157afa427ab80f473e879b703a"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "1548b98949ed5cacd4287dc0b2db2574"
server_key_iv: "fd5274e10c9b0067e204a33c99d57180"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.402736 17816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:23:58.405503 17838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.406922 17816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:23:58.407051 17816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "f4f8aa157afa427ab80f473e879b703a"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "1548b98949ed5cacd4287dc0b2db2574"
server_key_iv: "fd5274e10c9b0067e204a33c99d57180"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.407181 17816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:58.433879 17816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:58.437314 17816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:58.437506 17816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:58.444903 17816 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:36419
I20260811 22:23:58.444933 17919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:36419 every 8 connection(s)
I20260811 22:23:58.445924 17816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:58.449015 17923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:58.452989 17923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:23:58.454651 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 17816
I20260811 22:23:58.454762 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:23:58.455082 27124 external_mini_cluster.cc:1428] Setting key 3f6293a363c77686fe0257ea98f10f5e
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.61@KRBTEST.COM" created.
I20260811 22:23:58.469117 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.455464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40936 (local address 127.26.125.62:36419)
0811 22:23:58.456029 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.456048 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.456146 (+ 98us) server_negotiation.cc:407] Connection header received
0811 22:23:58.457218 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.457222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.457406 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.457575 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.458444 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.459368 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.460383 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.460755 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.460903 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.463590 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.463599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.463613 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.463638 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.465369 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.465954 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.465963 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.465964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.466122 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.466536 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.466540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.466541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.467211 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.467328 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.467899 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.468158 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"spinlock_wait_cycles":22912,"thread_start_us":151,"threads_started":1}
I20260811 22:23:58.469117 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.455463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36419 (local address 127.0.0.1:40936)
0811 22:23:58.455994 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.456046 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.456902 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.457600 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.457606 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.458049 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.458301 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.458314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.459480 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.459482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.460272 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.460283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.460903 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.460906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.461020 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.461702 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.461719 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.463401 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.465515 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.465525 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.465536 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.465825 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.466221 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.466224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.466225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.466416 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.467340 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.467349 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.467610 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.467819 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.468281 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":101,"thread_start_us":135,"threads_started":1}
I20260811 22:23:58.471987 17923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:23:58.472743 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.472465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56410)
0811 22:23:58.472598 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.472673 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:23:58.472954 17841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
I20260811 22:23:58.473865 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.473659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56426)
0811 22:23:58.473764 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.473810 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:23:58.474009 17923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:43297: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab.
Entry for principal kudu/127.26.125.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab.
I20260811 22:23:58.486093 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.61:43297
--webserver_interface=127.26.125.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab
--principal=kudu/127.26.125.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36419,127.26.125.61:43297,127.26.125.60:41243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:23:58.523262 17923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } attempt: 1
I20260811 22:23:58.524210 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.523890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56440)
0811 22:23:58.524059 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.524128 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 22:23:58.524957 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.524759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56454)
0811 22:23:58.524850 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.524897 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 22:23:58.525079 17923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:43297: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
I20260811 22:23:58.593250 17923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } attempt: 2
I20260811 22:23:58.594081 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.593782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56456)
0811 22:23:58.593926 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.593992 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:23:58.594960 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.594697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56460)
0811 22:23:58.594806 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.594887 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:23:58.595118 17923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:43297: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43297: connect: Connection refused (error 111)
W20260811 22:23:58.604916 17934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:58.605219 17934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:58.605342 17934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:58.609615 17934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:58.609692 17934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:58.609712 17934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:58.609728 17934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:58.609741 17934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:58.615487 17934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36419,127.26.125.61:43297,127.26.125.60:41243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab
--principal=kudu/127.26.125.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.61:43297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.125.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:58.616631 17934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:58.617663 17934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:58.626502 17947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.626591 17945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.626595 17943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:58.626747 17934 server_base.cc:1034] running on GCE node
I20260811 22:23:58.627570 17934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:58.628890 17934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:58.630093 17934 hybrid_clock.cc:648] HybridClock initialized: now 1786487038630055 us; error 50 us; skew 500 ppm
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:58.633262 17934 init.cc:377] Logged in from keytab as kudu/127.26.125.61@KRBTEST.COM (short username kudu)
I20260811 22:23:58.634320 17934 webserver.cc:460] Webserver started at http://127.26.125.61:34235/ using document root <none> and password file <none>
I20260811 22:23:58.634979 17934 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:58.635066 17934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:58.635290 17934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:58.637446 17934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/data/instance:
uuid: "9106c26b359d45d7b678c649230b3b62"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "6497850696d82a6730ae269856d11d1e"
server_key_iv: "bc734a5b4b4ca6916a2a9da94f6b2778"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.638029 17934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/wal/instance:
uuid: "9106c26b359d45d7b678c649230b3b62"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "6497850696d82a6730ae269856d11d1e"
server_key_iv: "bc734a5b4b4ca6916a2a9da94f6b2778"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.641975 17934 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:23:58.644469 17958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.645737 17934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:23:58.645864 17934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/wal
uuid: "9106c26b359d45d7b678c649230b3b62"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "6497850696d82a6730ae269856d11d1e"
server_key_iv: "bc734a5b4b4ca6916a2a9da94f6b2778"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.645982 17934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:58.665512 17934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:58.669087 17934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:58.669287 17934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:58.676512 17934 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.61:43297
I20260811 22:23:58.676533 18039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.61:43297 every 8 connection(s)
I20260811 22:23:58.677536 17934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
I20260811 22:23:58.680516 18040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:58.684473 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 17934
I20260811 22:23:58.684645 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-1/wal/instance
I20260811 22:23:58.684471 18040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:23:58.684945 27124 external_mini_cluster.cc:1428] Setting key 4ebdaf2cbcf2004d1a840cb27cfb3734
I20260811 22:23:58.686308 17923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } attempt: 3
WARNING: no policy specified for kudu/127.26.125.60@KRBTEST.COM; defaulting to no policy
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.60@KRBTEST.COM" created.
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:58.700407 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.686793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56472)
0811 22:23:58.686996 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.687014 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.687115 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.689477 (+ 2362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.689481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.689691 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.689824 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.689831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.692039 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.692043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.692853 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.692874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.693744 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.693747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.693845 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.694477 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.694491 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.696010 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.698060 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.698064 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.698067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.698301 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.698671 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.698674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.698677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.698803 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.699820 (+ 1017us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.699824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.699946 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.700070 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.700225 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"spinlock_wait_cycles":896}
I20260811 22:23:58.702262 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.687392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56472 (local address 127.26.125.61:43297)
0811 22:23:58.687970 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.687983 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.688027 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:23:58.688957 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.688984 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.689277 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.689984 (+ 707us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.690394 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.691903 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.692994 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.693616 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.693729 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.696170 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.696177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.696191 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.696215 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.697869 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.698398 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.698404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.698410 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.698581 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.698958 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.698962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.698964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.699690 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.699840 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.700551 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.701158 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":454,"spinlock_wait_cycles":16000,"thread_start_us":169,"threads_started":1}
I20260811 22:23:58.703526 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.687425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36419 (local address 127.0.0.1:40950)
0811 22:23:58.687914 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.687970 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.688974 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.689505 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.689516 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.689915 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.690570 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.690583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.691727 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.691731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.692899 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.692909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.693428 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.693433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.693649 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.694368 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.694395 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.699764 (+ 5369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.701681 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.701691 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.701698 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.701988 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.702287 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.702290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.702292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.702387 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.702894 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.702901 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.703024 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.703182 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.703366 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":53,"thread_start_us":114,"threads_started":1}
I20260811 22:23:58.704337 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.687260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.26.125.62:36419)
0811 22:23:58.687370 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.687373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.688068 (+ 695us) server_negotiation.cc:407] Connection header received
0811 22:23:58.689236 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.689239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.689316 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.689411 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.690780 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.691588 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.693025 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.693270 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.693356 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.699952 (+ 6596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.699955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.699957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.699982 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.701564 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.702097 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.702102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.702104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.702194 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.702502 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.702505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.702507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.702717 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.702787 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.704052 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.704197 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:23:58.704845 17923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:23:58.705521 18040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:23:58.705643 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.705326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34858)
0811 22:23:58.705518 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.705570 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 22:23:58.706434 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.706142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34860)
0811 22:23:58.706320 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.706369 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260811 22:23:58.706581 17923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:41243: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
I20260811 22:23:58.714911 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.705953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56482)
0811 22:23:58.706675 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.706690 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.706780 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.707033 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.707036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.707221 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.707343 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.707351 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.708032 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.708035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.708843 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.708855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.709296 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.709299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.709393 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.709782 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.709792 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.711985 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.713517 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.713520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.713522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.713706 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.713954 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.713958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.713960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.714069 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.714374 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.714376 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.714473 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.714559 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.714707 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652}
I20260811 22:23:58.714938 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.706162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.26.125.61:43297)
0811 22:23:58.706261 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.706265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.706755 (+ 490us) server_negotiation.cc:407] Connection header received
0811 22:23:58.706882 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.706884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.706931 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.706996 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.707441 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.707929 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.708961 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.709189 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.709263 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.712114 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.712118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.712119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.712142 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.713417 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.713788 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.713791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.713793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.713870 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.714139 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.714141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.714142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.714310 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.714379 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.714618 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.714780 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":34}
I20260811 22:23:58.715770 18040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:23:58.716423 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.716184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34866)
0811 22:23:58.716291 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.716367 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:23:58.716969 17959 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab.
Entry for principal kudu/127.26.125.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab.
I20260811 22:23:58.717881 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.717707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34872)
0811 22:23:58.717787 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.717830 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 22:23:58.718017 18040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:41243: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
I20260811 22:23:58.718220 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.60:41243
--webserver_interface=127.26.125.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab
--principal=kudu/127.26.125.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36419,127.26.125.61:43297,127.26.125.60:41243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:23:58.737772 17923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } attempt: 1
I20260811 22:23:58.738749 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.738419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34876)
0811 22:23:58.738587 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.738657 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:23:58.739518 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.739292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34878)
0811 22:23:58.739393 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.739455 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 22:23:58.739662 17923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:41243: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
I20260811 22:23:58.767277 18040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } attempt: 1
I20260811 22:23:58.768222 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.767885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34880)
0811 22:23:58.768047 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.768125 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:23:58.768913 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.768691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34884)
0811 22:23:58.768785 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.768836 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 22:23:58.769032 18040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:41243: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
I20260811 22:23:58.814814 17923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } attempt: 2
I20260811 22:23:58.815690 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.815384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34890)
0811 22:23:58.815536 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.815606 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:23:58.816490 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.816270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34898)
0811 22:23:58.816377 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.816424 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:23:58.816635 17923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:41243: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
I20260811 22:23:58.837194 18040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } attempt: 2
I20260811 22:23:58.838045 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.837733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34906)
0811 22:23:58.837884 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.837953 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:23:58.838718 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.838493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34912)
0811 22:23:58.838599 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.838652 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 22:23:58.838835 18040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:41243: Network error: Client connection negotiation failed: client connection to 127.26.125.60:41243: connect: Connection refused (error 111)
W20260811 22:23:58.838573 18051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:58.838964 18051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:58.839020 18051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:58.843067 18051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:58.843124 18051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:58.843143 18051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:58.843158 18051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:58.843171 18051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:58.848650 18051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36419,127.26.125.61:43297,127.26.125.60:41243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab
--principal=kudu/127.26.125.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.60:41243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.125.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:23:58.849838 18051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:58.851018 18051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:58.859826 18062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:58.859898 18051 server_base.cc:1034] running on GCE node
W20260811 22:23:58.859827 18063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.859843 18065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:58.860780 18051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:58.862011 18051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:58.863196 18051 hybrid_clock.cc:648] HybridClock initialized: now 1786487038863173 us; error 39 us; skew 500 ppm
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:58.866214 18051 init.cc:377] Logged in from keytab as kudu/127.26.125.60@KRBTEST.COM (short username kudu)
I20260811 22:23:58.867486 18051 webserver.cc:460] Webserver started at http://127.26.125.60:35183/ using document root <none> and password file <none>
I20260811 22:23:58.868103 18051 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:58.868175 18051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:58.868393 18051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:58.870620 18051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/data/instance:
uuid: "327b3263dbe24bb68821582f53b34e59"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "e09e2770ff42f985c47365439d6fbdb4"
server_key_iv: "0cca08499f4bba2d2b24d1fdb4f53661"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.871258 18051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/wal/instance:
uuid: "327b3263dbe24bb68821582f53b34e59"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "e09e2770ff42f985c47365439d6fbdb4"
server_key_iv: "0cca08499f4bba2d2b24d1fdb4f53661"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.875269 18051 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:23:58.877835 18075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.879236 18051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:23:58.879369 18051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/wal
uuid: "327b3263dbe24bb68821582f53b34e59"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-5bn2"
server_key: "e09e2770ff42f985c47365439d6fbdb4"
server_key_iv: "0cca08499f4bba2d2b24d1fdb4f53661"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.879477 18051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:58.900768 18051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:58.904327 18051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:58.904529 18051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:58.912878 18051 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.60:41243
I20260811 22:23:58.912885 18158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.60:41243 every 8 connection(s)
I20260811 22:23:58.914340 18051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
I20260811 22:23:58.915787 17923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } attempt: 3
I20260811 22:23:58.916106 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 18051
I20260811 22:23:58.916231 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486868287078-27124-0/minicluster-data/master-2/wal/instance
I20260811 22:23:58.916522 27124 external_mini_cluster.cc:1428] Setting key cab40d5ad568d3afee594f69b745979e
I20260811 22:23:58.919353 18159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:58.924448 18159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
I20260811 22:23:58.929993 18040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } attempt: 3
I20260811 22:23:58.932744 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.916302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34924)
0811 22:23:58.916520 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.916538 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.916638 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.920244 (+ 3606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.920247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.920473 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.920614 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.920623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.923121 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.923126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.923962 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.923978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.925113 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.925117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.925215 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.925817 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.925835 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.927670 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.930611 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.930615 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.930617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.930897 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.931298 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.931302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.931304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.931403 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.932223 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.932227 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.932333 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.932419 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.932561 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:58.934667 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.917391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34924 (local address 127.26.125.60:41243)
0811 22:23:58.919140 (+ 1749us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.919148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.919163 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:23:58.919365 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.919389 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.919874 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.920847 (+ 973us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.921371 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.922944 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.924135 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.924949 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.925574 (+ 625us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.927862 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.927869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.927886 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.927917 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.930502 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.931021 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.931027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.931033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.931190 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.931511 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.931514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.931516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.932093 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.933092 (+ 999us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.933268 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.933585 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":438,"server-negotiator.queue_time_us":1649,"spinlock_wait_cycles":11648,"thread_start_us":145,"threads_started":1}
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
I20260811 22:23:58.939324 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.927938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.26.125.62:36419)
0811 22:23:58.928066 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.928071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.928086 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:23:58.928123 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.928126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.928169 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.928241 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.929291 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.930057 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.931164 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.931468 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.931571 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.934482 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.934486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.934489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.934518 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.936570 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.937446 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.937450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.937452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.937551 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.938128 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.938132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.938133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.938472 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.938551 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.939051 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.939172 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:23:58.939926 17923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a: Bootstrap starting.
I20260811 22:23:58.940152 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.927020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36419 (local address 127.0.0.1:40962)
0811 22:23:58.927383 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.927941 (+ 558us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.928046 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.928381 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.928388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.929008 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.929154 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.929163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.930216 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.930219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.931053 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.931063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.931704 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.931707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.931808 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.932337 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.932352 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.934345 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.936760 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.936770 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.936777 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.937115 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.937676 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.937679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.937681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.938005 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.938629 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.938635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.938761 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.939710 (+ 949us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.939902 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260811 22:23:58.940294 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.930466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34930)
0811 22:23:58.930608 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.930623 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.930696 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.931214 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.931217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.931392 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.931499 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.931507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.933343 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.933346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.934159 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.934173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.934893 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.934898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.934994 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.935441 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.935464 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.936834 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.938623 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.938626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.938628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.938892 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.939300 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.939304 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.939306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.939398 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.939831 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.939834 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.939936 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.940016 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.940144 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 22:23:58.940822 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.930708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.26.125.60:41243)
0811 22:23:58.931018 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.931026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.931037 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:23:58.931064 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.931067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.931104 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.931188 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.931619 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.933217 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.934340 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.934729 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.935271 (+ 542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.936975 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.936979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.936981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.937006 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.938511 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.939001 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.939004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.939005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.939196 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.939530 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.939533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.939534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.939726 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.940450 (+ 724us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.940539 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.940651 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":243,"thread_start_us":97,"threads_started":1}
I20260811 22:23:58.941833 18159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:23:58.943418 17923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:58.943934 18040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62: Bootstrap starting.
I20260811 22:23:58.944355 17923 log.cc:824] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
I20260811 22:23:58.946722 17923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a: No bootstrap required, opened a new log
I20260811 22:23:58.946717 18040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:58.947631 18040 log.cc:824] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:58.949949 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.942349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.26.125.61:43297)
0811 22:23:58.942450 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.942453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.942585 (+ 132us) server_negotiation.cc:407] Connection header received
0811 22:23:58.942684 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.942687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.942723 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.942777 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.943207 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.943740 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.944568 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.944809 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.944886 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.946937 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.946941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.946944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.946977 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.948367 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.948756 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.948760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.948762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.948838 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.949084 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.949087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.949089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.949310 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.949394 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.949720 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.949822 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:23:58.949944 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.942283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43297 (local address 127.0.0.1:56484)
0811 22:23:58.942516 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.942527 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.942605 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.942803 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.942806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.943005 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.943106 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.943111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.943838 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.943841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.944431 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.944441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.945049 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.945052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.945147 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.945628 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.945642 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.946748 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.948489 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.948491 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.948493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.948666 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.948918 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.948922 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.948923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.949003 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.949397 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.949400 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.949537 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.949632 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.949773 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260811 22:23:58.951000 18159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:23:58.951013 17923 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:58.951275 17923 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:58.951330 17923 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f8aa157afa427ab80f473e879b703a, State: Initialized, Role: FOLLOWER
I20260811 22:23:58.952854 17923 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:58.953485 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.917391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.26.125.60:41243)
0811 22:23:58.918008 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.918029 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.918385 (+ 356us) server_negotiation.cc:407] Connection header received
0811 22:23:58.919361 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.919387 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.919874 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.921102 (+ 1228us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.921370 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.922943 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.924104 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.924946 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.925571 (+ 625us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.928171 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.928174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.928177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.928200 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.930265 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.944804 (+ 14539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.944808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.944810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.944901 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.951970 (+ 7069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.951973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.951975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.952200 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.952392 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.953168 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.953287 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":462,"spinlock_wait_cycles":6912,"thread_start_us":139,"threads_started":1}
I20260811 22:23:58.954388 17923 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:58.954792 18040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62: No bootstrap required, opened a new log
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
I20260811 22:23:58.958992 18040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:58.959231 18040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:58.959295 18040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9106c26b359d45d7b678c649230b3b62, State: Initialized, Role: FOLLOWER
I20260811 22:23:58.959821 18040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:58.961045 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.951574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.26.125.60:41243)
0811 22:23:58.951690 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.951695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.951709 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:23:58.951745 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.951748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.951790 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.951874 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.952175 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.952854 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.954038 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.954320 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.954379 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.957715 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.957718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.957721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.957743 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.959360 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.959756 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.959759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.959761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.959836 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.960107 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.960110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.960111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.960321 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.960397 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.960714 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.960838 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:23:58.961301 18040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:58.961524 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.951517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:41243 (local address 127.0.0.1:34940)
0811 22:23:58.951609 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.951622 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.951690 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.951880 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.951882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.952006 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.952078 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.952082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.952977 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.952979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.953925 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.953933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.954670 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.954673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.954748 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.956134 (+ 1386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:58.956149 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.957599 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.959455 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.959457 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.959460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.959674 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.959922 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:58.959925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:58.959927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:58.960021 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:58.960456 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.960459 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.960550 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.961243 (+ 693us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.961376 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:23:58.959656 17816 master_runner.cc:186] Error getting master registration for 127.26.125.61:43297: OK, instance_id { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" instance_seqno: 1786487038674422 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:23:58.963732 17816 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:23:58.965080 18159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59: Bootstrap starting.
I20260811 22:23:58.967244 18194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:58.967415 18194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:23:58.967818 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.955142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.26.125.62:36419)
0811 22:23:58.955275 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.955280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.955558 (+ 278us) server_negotiation.cc:407] Connection header received
0811 22:23:58.955600 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.955602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.955640 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.955702 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.956753 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.957514 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.959305 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.959634 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.959734 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.963254 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.963259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.963262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.963297 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.965130 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.965769 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.965773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.965775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.965872 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.966200 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.966203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.966205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.966445 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.966674 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.967374 (+ 700us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.967659 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:23:58.968484 18159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:58.969432 18159 log.cc:824] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59: Log is configured to *not* fsync() on all Append() calls
W20260811 22:23:58.970477 18191 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:23:58.970571 18191 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:23:58.971776 18175 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:58.971939 18175 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:23:58.971973 18159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59: No bootstrap required, opened a new log
Aug 11 22:23:58 dist-test-slave-5bn2 krb5kdc[17800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
W20260811 22:23:58.978304 18220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:23:58.978397 18220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:23:58.980269 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.969485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.26.125.61:43297)
0811 22:23:58.969615 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.969619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.969637 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:23:58.969785 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.969788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.969838 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.969918 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.970954 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.971663 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.972917 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.973229 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.973306 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.976396 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.976400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.976403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:58.976429 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.978102 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.978687 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.978691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.978693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.978785 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:58.979158 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:58.979162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:58.979164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:58.979413 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.979594 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.979977 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.980108 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:23:58.980928 18159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:58.981199 18159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:58.981256 18159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 327b3263dbe24bb68821582f53b34e59, State: Initialized, Role: FOLLOWER
I20260811 22:23:58.981730 18159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:58.982234 18223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:58.982425 18223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:23:58.983419 18159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:23:58.988575 18237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:23:58.988657 18237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:23:58.980657 17934 master_runner.cc:186] Error getting master registration for 127.26.125.60:41243: OK, instance_id { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" instance_seqno: 1786487038910249 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:23:58.993047 17934 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:23:58.996140 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:23:58.998430 18051 master_runner.cc:263] No leader master found from master 327b3263dbe24bb68821582f53b34e59
I20260811 22:23:58.998533 18051 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:23:59.015226 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.003398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.26.125.62:36419)
0811 22:23:59.003548 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.003553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.005185 (+ 1632us) server_negotiation.cc:407] Connection header received
0811 22:23:59.005324 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.005328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.005386 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.005479 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:59.006793 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.007684 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.008977 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.009322 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.009396 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.011413 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:59.011417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:59.011420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:59.011463 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:59.013232 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:59.013755 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:59.013759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:59.013761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:59.013850 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:59.014135 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:59.014138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:59.014140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:59.014371 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:59.014504 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.014900 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.015041 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:23:59.021315 18223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:59.021557 18223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:59.022737 18223 leader_election.cc:288] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4f8aa157afa427ab80f473e879b703a (127.26.125.62:36419), 9106c26b359d45d7b678c649230b3b62 (127.26.125.61:43297)
I20260811 22:23:59.023759 17883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "327b3263dbe24bb68821582f53b34e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4f8aa157afa427ab80f473e879b703a" is_pre_election: true
I20260811 22:23:59.024132 17883 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 327b3263dbe24bb68821582f53b34e59 in term 0.
I20260811 22:23:59.024601 18001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "327b3263dbe24bb68821582f53b34e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9106c26b359d45d7b678c649230b3b62" is_pre_election: true
I20260811 22:23:59.024924 18001 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 327b3263dbe24bb68821582f53b34e59 in term 0.
I20260811 22:23:59.025396 18076 leader_election.cc:302] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 327b3263dbe24bb68821582f53b34e59, 9106c26b359d45d7b678c649230b3b62; no voters:
I20260811 22:23:59.026124 18223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:59.026222 18223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:59.026271 18223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:59.027668 18223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:59.028415 17883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "327b3263dbe24bb68821582f53b34e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4f8aa157afa427ab80f473e879b703a"
I20260811 22:23:59.028575 17883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:59.029982 17883 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 327b3263dbe24bb68821582f53b34e59 in term 1.
I20260811 22:23:59.030391 18077 leader_election.cc:302] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 327b3263dbe24bb68821582f53b34e59, f4f8aa157afa427ab80f473e879b703a; no voters:
I20260811 22:23:59.031167 18001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "327b3263dbe24bb68821582f53b34e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9106c26b359d45d7b678c649230b3b62"
I20260811 22:23:59.031317 18001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:59.032635 18001 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 327b3263dbe24bb68821582f53b34e59 in term 1.
I20260811 22:23:59.033078 18223 leader_election.cc:288] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [CANDIDATE]: Term 1 election: Requested vote from peers f4f8aa157afa427ab80f473e879b703a (127.26.125.62:36419), 9106c26b359d45d7b678c649230b3b62 (127.26.125.61:43297)
I20260811 22:23:59.033304 18223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:59.033394 18223 raft_consensus.cc:686] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [term 1 LEADER]: Becoming Leader. State: Replica: 327b3263dbe24bb68821582f53b34e59, State: Running, Role: LEADER
I20260811 22:23:59.033778 18223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } }
I20260811 22:23:59.035899 18223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 327b3263dbe24bb68821582f53b34e59. Latest consensus state: current_term: 1 leader_uuid: "327b3263dbe24bb68821582f53b34e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:59.036033 18223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:59.047996 18254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:59.049690 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.003975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.26.125.61:43297)
0811 22:23:59.004088 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.004092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.005591 (+ 1499us) server_negotiation.cc:407] Connection header received
0811 22:23:59.005643 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.005645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.005694 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.005764 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:59.018727 (+ 12963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.019517 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.037127 (+ 17610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.037512 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.037605 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.040683 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:59.040688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:59.040691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:59.040726 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:59.042557 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:59.045915 (+ 3358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:59.045919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:59.045928 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:59.046026 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:59.046948 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:59.046952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:59.046954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:59.047189 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:59.047343 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.049349 (+ 2006us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.049519 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:23:59.051828 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.004560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34952 (local address 127.26.125.60:41243)
0811 22:23:59.004670 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.004674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.005761 (+ 1087us) server_negotiation.cc:407] Connection header received
0811 22:23:59.005923 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.005927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.005990 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.006044 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:59.006946 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.007694 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.020673 (+ 12979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.021057 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.021140 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.043175 (+ 22035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:59.043179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:59.043182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:59.043215 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:59.044998 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:59.045514 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:59.045518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:59.045520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:59.045610 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:59.046333 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:59.046338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:59.046340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:59.046598 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:59.046715 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.051530 (+ 4815us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.051668 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:23:59.055536 18254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:59.066715 18001 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [term 1 FOLLOWER]: Refusing update from remote peer 327b3263dbe24bb68821582f53b34e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:59.067497 18223 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:59.072494 18194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 327b3263dbe24bb68821582f53b34e59. Latest consensus state: current_term: 1 leader_uuid: "327b3263dbe24bb68821582f53b34e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:59.072726 18194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:23:59.075867 17883 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [term 1 FOLLOWER]: Refusing update from remote peer 327b3263dbe24bb68821582f53b34e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:59.076601 18223 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:59.086979 18175 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 327b3263dbe24bb68821582f53b34e59. Latest consensus state: current_term: 1 leader_uuid: "327b3263dbe24bb68821582f53b34e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:59.087145 18175 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:23:59.095376 18249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "327b3263dbe24bb68821582f53b34e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:59.095561 18249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:59.096701 18194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "327b3263dbe24bb68821582f53b34e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:59.096853 18194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:23:59.097496 18256 mvcc.cc:205] Tried to move back new op lower bound from 7317450911994576896 to 7317450911886409728. Current Snapshot: MvccSnapshot[applied={T|T < 7317450911994576896}]
I20260811 22:23:59.098838 18257 mvcc.cc:205] Tried to move back new op lower bound from 7317450911994576896 to 7317450911886409728. Current Snapshot: MvccSnapshot[applied={T|T < 7317450911994576896}]
I20260811 22:23:59.099779 18223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "327b3263dbe24bb68821582f53b34e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:59.099953 18223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:59.103732 18175 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "327b3263dbe24bb68821582f53b34e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f8aa157afa427ab80f473e879b703a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36419 } } peers { permanent_uuid: "9106c26b359d45d7b678c649230b3b62" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43297 } } peers { permanent_uuid: "327b3263dbe24bb68821582f53b34e59" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 41243 } } }
I20260811 22:23:59.103888 18175 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:23:59.104682 18260 mvcc.cc:205] Tried to move back new op lower bound from 7317450911994576896 to 7317450911886409728. Current Snapshot: MvccSnapshot[applied={T|T < 7317450911994576896}]
I20260811 22:23:59.104974 18254 catalog_manager.cc:1347] Generated new cluster ID: 4384e282c7524aefbde835f82bc3690a
I20260811 22:23:59.105033 18254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:59.140411 18254 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:59.141813 18254 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:59.164454 18254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 327b3263dbe24bb68821582f53b34e59: Generated new TSK 0
I20260811 22:23:59.165560 18254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:23:59.972138 18191 catalog_manager.cc:1259] Loaded cluster ID: 4384e282c7524aefbde835f82bc3690a
I20260811 22:23:59.972235 18191 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a: loading cluster ID for follower catalog manager: success
I20260811 22:23:59.973937 18191 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a: acquiring CA information for follower catalog manager: success
I20260811 22:23:59.975198 18191 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f4f8aa157afa427ab80f473e879b703a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:23:59.979713 18220 catalog_manager.cc:1259] Loaded cluster ID: 4384e282c7524aefbde835f82bc3690a
I20260811 22:23:59.979782 18220 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62: loading cluster ID for follower catalog manager: success
I20260811 22:23:59.981386 18220 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62: acquiring CA information for follower catalog manager: success
I20260811 22:23:59.982548 18220 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9106c26b359d45d7b678c649230b3b62: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:24:00.250604 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 17816
I20260811 22:24:00.263514 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 17934
I20260811 22:24:00.283397 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 18051
2026-08-11T22:24:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3378 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:00 dist-test-slave-5bn2 krb5kdc[18273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:00 dist-test-slave-5bn2 krb5kdc[18273](info): set up 2 sockets
Aug 11 22:24:00 dist-test-slave-5bn2 krb5kdc[18273](info): commencing operation
krb5kdc: starting...
W20260811 22:24:01.485481 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:24:01 dist-test-slave-5bn2 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:24:01.623724 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44291
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:36203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:44291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:01.740263 18294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:01.740568 18294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:01.740625 18294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:01.744783 18294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:01.744869 18294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:01.744895 18294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:01.744917 18294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:01.744947 18294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:01.750769 18294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:44291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:01.751958 18294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:01.753093 18294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:01.762228 18302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:01.762233 18300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:01.762233 18306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:01.762513 18294 server_base.cc:1034] running on GCE node
I20260811 22:24:01.763319 18294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:01.764539 18294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:01.765709 18294 hybrid_clock.cc:648] HybridClock initialized: now 1786487041765685 us; error 39 us; skew 500 ppm
Aug 11 22:24:01 dist-test-slave-5bn2 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487041, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:01.768754 18294 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:24:01.769767 18294 webserver.cc:460] Webserver started at http://127.26.125.62:45021/ using document root <none> and password file <none>
I20260811 22:24:01.770387 18294 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:01.770462 18294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:01.770689 18294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:01.772922 18294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "34abe35a8c624f918b9923258732500d"
format_stamp: "Formatted at 2026-08-11 22:24:01 on dist-test-slave-5bn2"
server_key: "766293c8c14620f7d939cb485bc43a5f"
server_key_iv: "f31d3b67cc053e2cd4ee56e8b9591a45"
server_key_version: "encryptionkey@0"
I20260811 22:24:01.773505 18294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "34abe35a8c624f918b9923258732500d"
format_stamp: "Formatted at 2026-08-11 22:24:01 on dist-test-slave-5bn2"
server_key: "766293c8c14620f7d939cb485bc43a5f"
server_key_iv: "f31d3b67cc053e2cd4ee56e8b9591a45"
server_key_version: "encryptionkey@0"
I20260811 22:24:01.777401 18294 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 22:24:01.779965 18316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:01.781227 18294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:24:01.781349 18294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "34abe35a8c624f918b9923258732500d"
format_stamp: "Formatted at 2026-08-11 22:24:01 on dist-test-slave-5bn2"
server_key: "766293c8c14620f7d939cb485bc43a5f"
server_key_iv: "f31d3b67cc053e2cd4ee56e8b9591a45"
server_key_version: "encryptionkey@0"
I20260811 22:24:01.781453 18294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:01.794167 18294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:01.797930 18294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:01.798161 18294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:01.805466 18294 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:44291
I20260811 22:24:01.805491 18395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:44291 every 8 connection(s)
I20260811 22:24:01.806476 18294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:01.809484 18396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:01.811532 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 18294
I20260811 22:24:01.811664 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:24:01.811956 27124 external_mini_cluster.cc:1428] Setting key 5c48b9e2eb6c0addf313e16271ee1075
I20260811 22:24:01.816676 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d: Bootstrap starting.
I20260811 22:24:01.819566 18396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:01.820376 18396 log.cc:824] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:24:01 dist-test-slave-5bn2 krb5kdc[18273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:01.822546 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d: No bootstrap required, opened a new log
I20260811 22:24:01.826812 18396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abe35a8c624f918b9923258732500d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44291 } }
I20260811 22:24:01.827100 18396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:01.827162 18396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34abe35a8c624f918b9923258732500d, State: Initialized, Role: FOLLOWER
I20260811 22:24:01.827349 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:01.813330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.26.125.62:44291)
0811 22:24:01.813792 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:01.813812 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:01.813859 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:24:01.814653 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:01.814680 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:01.815172 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:01.815566 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:01.816267 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:01.817808 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:01.818731 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:01.819493 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:01.819663 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:01.822193 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:01.822199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:01.822215 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:01.822246 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:01.823919 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:01.824484 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:01.824491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:01.824498 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:01.824671 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:01.825033 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:01.825037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:01.825038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:01.825608 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:01.825776 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:01.826110 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 22:24:01.826436 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":118,"threads_started":1}
I20260811 22:24:01.827661 18396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abe35a8c624f918b9923258732500d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44291 } }
I20260811 22:24:01.827826 18396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:01.827894 18396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:01.827994 18396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:01.829031 18396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abe35a8c624f918b9923258732500d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44291 } }
I20260811 22:24:01.829377 18396 leader_election.cc:302] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34abe35a8c624f918b9923258732500d; no voters:
I20260811 22:24:01.829732 18396 leader_election.cc:288] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:01.829847 18403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:01.830076 18403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [term 1 LEADER]: Becoming Leader. State: Replica: 34abe35a8c624f918b9923258732500d, State: Running, Role: LEADER
I20260811 22:24:01.830444 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:01.830468 18403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abe35a8c624f918b9923258732500d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44291 } }
I20260811 22:24:01.831127 18396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:01.832315 18404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34abe35a8c624f918b9923258732500d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abe35a8c624f918b9923258732500d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44291 } } }
I20260811 22:24:01.832436 18404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:01.832517 18409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34abe35a8c624f918b9923258732500d. Latest consensus state: current_term: 1 leader_uuid: "34abe35a8c624f918b9923258732500d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34abe35a8c624f918b9923258732500d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44291 } } }
I20260811 22:24:01.832691 18409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:01.832824 18416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:01.836216 18416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:01.842188 18416 catalog_manager.cc:1347] Generated new cluster ID: 69382f461f4a4543b400fdbe6d1fd109
I20260811 22:24:01.842269 18416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:01.851642 18416 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:01.852650 18416 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:01.859153 18416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34abe35a8c624f918b9923258732500d: Generated new TSK 0
I20260811 22:24:01.859965 18416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:01.888286 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 18294
2026-08-11T22:24:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1607 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:01 dist-test-slave-5bn2 krb5kdc[18437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:01 dist-test-slave-5bn2 krb5kdc[18437](info): set up 2 sockets
Aug 11 22:24:01 dist-test-slave-5bn2 krb5kdc[18437](info): commencing operation
krb5kdc: starting...
W20260811 22:24:03.079483 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:24:03 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:24:03.215637 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36329
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:03.337662 18457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:03.337981 18457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:03.338040 18457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:03.342288 18457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:03.342375 18457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:03.342406 18457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:03.342427 18457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:03.342453 18457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:03.348145 18457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:03.349326 18457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:03.350442 18457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:03.359359 18476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:03.359355 18468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:03.359355 18467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:03.359402 18457 server_base.cc:1034] running on GCE node
I20260811 22:24:03.360332 18457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:03.361541 18457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:03.362742 18457 hybrid_clock.cc:648] HybridClock initialized: now 1786487043362707 us; error 50 us; skew 500 ppm
Aug 11 22:24:03 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487043, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:03.370659 18457 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:24:03.371796 18457 webserver.cc:460] Webserver started at http://127.26.125.62:41575/ using document root <none> and password file <none>
I20260811 22:24:03.372423 18457 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:03.372493 18457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:03.372717 18457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:03.375130 18457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "d4d59c48f5d24a46b2aa332d90de4eed"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "4e90f89d3d4ee61d5f9a0a59fccc09d7"
server_key_iv: "f37e47d0ab01f700659d4dab21085641"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.375758 18457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "d4d59c48f5d24a46b2aa332d90de4eed"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "4e90f89d3d4ee61d5f9a0a59fccc09d7"
server_key_iv: "f37e47d0ab01f700659d4dab21085641"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.379890 18457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:24:03.382654 18487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:03.384125 18457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:24:03.384277 18457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "d4d59c48f5d24a46b2aa332d90de4eed"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "4e90f89d3d4ee61d5f9a0a59fccc09d7"
server_key_iv: "f37e47d0ab01f700659d4dab21085641"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.384397 18457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:03.404523 18457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:03.407979 18457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:03.408182 18457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:03.415372 18457 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:36329
I20260811 22:24:03.415700 18568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:36329 every 8 connection(s)
I20260811 22:24:03.416455 18457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:03.419523 18569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:03.424083 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 18457
I20260811 22:24:03.424214 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:24:03.424463 27124 external_mini_cluster.cc:1428] Setting key 64bad2b71764cc3775b02073d6e623fd
I20260811 22:24:03.425952 18569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Bootstrap starting.
I20260811 22:24:03.428932 18569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:03.429867 18569 log.cc:824] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:03.431988 18569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: No bootstrap required, opened a new log
Aug 11 22:24:03 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:03.435791 18569 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:03.436055 18569 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:03.436134 18569 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d59c48f5d24a46b2aa332d90de4eed, State: Initialized, Role: FOLLOWER
I20260811 22:24:03.436615 18569 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:03.436755 18569 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:03.436825 18569 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:03.436928 18569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:03.437875 18569 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:03.438241 18569 leader_election.cc:302] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4d59c48f5d24a46b2aa332d90de4eed; no voters:
I20260811 22:24:03.438609 18569 leader_election.cc:288] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:03.438719 18577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:03.438966 18577 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 1 LEADER]: Becoming Leader. State: Replica: d4d59c48f5d24a46b2aa332d90de4eed, State: Running, Role: LEADER
I20260811 22:24:03.439347 18577 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:03.439589 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:03.425876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.26.125.62:36329)
0811 22:24:03.426339 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:03.426361 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:03.426408 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:24:03.427312 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:03.427342 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:03.427806 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:03.428204 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:03.428925 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.430441 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.431355 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.431851 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.432025 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:03.434394 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:03.434399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:03.434414 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:03.434443 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:03.436263 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:03.436700 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:03.436707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:03.436712 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:03.436869 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:03.437189 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:03.437193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:03.437195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:03.437654 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:03.437843 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:03.438144 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 22:24:03.438491 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":138,"threads_started":1}
I20260811 22:24:03.439957 18569 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:03.441437 18580 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4d59c48f5d24a46b2aa332d90de4eed. Latest consensus state: current_term: 1 leader_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } } }
I20260811 22:24:03.441579 18580 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:03.442467 18589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:03.442448 18579 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } } }
I20260811 22:24:03.442986 18579 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:03.445490 18589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:03.452149 18589 catalog_manager.cc:1347] Generated new cluster ID: 6a479ffbbd6246bd94658ddd0c8e5c8e
I20260811 22:24:03.452243 18589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:03.460529 18589 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:03.461949 18589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:03.473275 18589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Generated new TSK 0
I20260811 22:24:03.474282 18589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:24:03.506386 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:36329
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:03.628010 18607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:03.628335 18607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:03.628396 18607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:03.632496 18607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:03.632582 18607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:03.632666 18607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:24:03.638425 18607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:03.639724 18607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:03.640900 18607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:03.651226 18616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:03.651225 18621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:03.651280 18615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:03.651679 18607 server_base.cc:1034] running on GCE node
I20260811 22:24:03.652132 18607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:03.652796 18607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:03.653962 18607 hybrid_clock.cc:648] HybridClock initialized: now 1786487043653941 us; error 31 us; skew 500 ppm
Aug 11 22:24:03 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487043, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:03.656903 18607 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:24:03.657836 18607 webserver.cc:460] Webserver started at http://127.26.125.1:40905/ using document root <none> and password file <none>
I20260811 22:24:03.658385 18607 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:03.658445 18607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:03.658619 18607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:03.660795 18607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "f2c1f08b3435460ea2ea7b91ad846a78"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "d9820c319a832f1f4d7988cb702c990e"
server_key_iv: "50a06d4ebd22295bbe49587effdf7f80"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.661330 18607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "f2c1f08b3435460ea2ea7b91ad846a78"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "d9820c319a832f1f4d7988cb702c990e"
server_key_iv: "50a06d4ebd22295bbe49587effdf7f80"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.665313 18607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:24:03.668004 18631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:03.669401 18607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:03.669518 18607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "f2c1f08b3435460ea2ea7b91ad846a78"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "d9820c319a832f1f4d7988cb702c990e"
server_key_iv: "50a06d4ebd22295bbe49587effdf7f80"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.669649 18607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:03.681818 18607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:03.685190 18607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:03.685395 18607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:03.686043 18607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:03.686987 18607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:03.687039 18607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:03.687083 18607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:03.687096 18607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:03.696966 18607 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:46629
I20260811 22:24:03.696990 18794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:46629 every 8 connection(s)
I20260811 22:24:03.697956 18607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:24:03.705116 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 18607
I20260811 22:24:03.705291 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:24:03.705646 27124 external_mini_cluster.cc:1428] Setting key f3a8261bb0a905356753a2e15a06b324
Aug 11 22:24:03 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487043, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:03.714102 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:03.700135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:43557 (local address 127.26.125.62:36329)
0811 22:24:03.700290 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:03.700294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:03.701329 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 22:24:03.702792 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:03.702796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:03.702885 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:03.702997 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:03.704455 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.705037 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.706761 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.707126 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.707231 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:03.710338 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:03.710342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:03.710345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:03.710385 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:03.711848 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:03.712396 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:03.712400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:03.712401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:03.712495 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:03.712856 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:03.712860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:03.712862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:03.713082 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:03.713167 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:03.713843 (+ 676us) server_negotiation.cc:299] Negotiation successful
0811 22:24:03.713970 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:24:03.715463 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:03.700549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:43557)
0811 22:24:03.701140 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:03.701193 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:03.702558 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:03.703226 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:03.703234 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:03.703716 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:03.704280 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:03.704302 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.705212 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.705217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:03.706625 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:03.706639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.707341 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.707348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:03.707564 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:03.708263 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:03.708286 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:03.710112 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:03.711976 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:03.711984 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:03.712001 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:03.712289 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:03.712598 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:03.712601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:03.712602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:03.712761 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:03.713199 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:03.713207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:03.713541 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 22:24:03.713779 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 22:24:03.714194 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":134,"threads_started":1}
I20260811 22:24:03.717034 18795 heartbeater.cc:347] Connected to a master server at 127.26.125.62:36329
I20260811 22:24:03.717311 18795 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:24:03.717967 18795 heartbeater.cc:511] Master 127.26.125.62:36329 requested a full tablet report, sending...
I20260811 22:24:03.719740 18519 ts_manager.cc:194] Registered new tserver with Master: f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:24:03.721097 18519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:43557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:24:03.736197 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:36329
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:03.854355 18802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:03.854665 18802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:03.854709 18802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:03.858866 18802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:03.858951 18802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:03.859022 18802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:24:03.864610 18802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:03.865782 18802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:03.866904 18802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:03.876922 18815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:03.876998 18813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:03.876924 18818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:03.876974 18802 server_base.cc:1034] running on GCE node
I20260811 22:24:03.877591 18802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:03.878244 18802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:03.879419 18802 hybrid_clock.cc:648] HybridClock initialized: now 1786487043879347 us; error 89 us; skew 500 ppm
Aug 11 22:24:03 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487043, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:03.883584 18802 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:24:03.884797 18802 webserver.cc:460] Webserver started at http://127.26.125.2:39327/ using document root <none> and password file <none>
I20260811 22:24:03.885594 18802 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:03.885692 18802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:03.885921 18802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:03.888165 18802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "44f9ae43d7974caca32cf1b2608cfbb5"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "cce361a0de9f2f647c58489b2f140783"
server_key_iv: "54b42fd8f8e6f13213d901e63cf5c160"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.888739 18802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "44f9ae43d7974caca32cf1b2608cfbb5"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "cce361a0de9f2f647c58489b2f140783"
server_key_iv: "54b42fd8f8e6f13213d901e63cf5c160"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.892889 18802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:24:03.895694 18830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:03.897162 18802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:24:03.897287 18802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "44f9ae43d7974caca32cf1b2608cfbb5"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "cce361a0de9f2f647c58489b2f140783"
server_key_iv: "54b42fd8f8e6f13213d901e63cf5c160"
server_key_version: "encryptionkey@0"
I20260811 22:24:03.897404 18802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:03.944245 18802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:03.947829 18802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:03.948014 18802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:03.948626 18802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:03.949508 18802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:03.949556 18802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:03.949599 18802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:03.949612 18802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:03.958503 18802 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:34653
I20260811 22:24:03.958542 19009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:34653 every 8 connection(s)
I20260811 22:24:03.959525 18802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:24:03.964746 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 18802
I20260811 22:24:03.964855 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:24:03.965107 27124 external_mini_cluster.cc:1428] Setting key e6c94b8af4b5054e567262b1053e2da9
Aug 11 22:24:03 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487043, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:03.974493 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:03.961407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:42983 (local address 127.26.125.62:36329)
0811 22:24:03.961554 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:03.961558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:03.962534 (+ 976us) server_negotiation.cc:407] Connection header received
0811 22:24:03.964002 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:03.964006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:03.964057 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:03.964136 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:03.965421 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.965968 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.967321 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.967635 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.967748 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:03.970349 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:03.970354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:03.970356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:03.970385 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:03.972045 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:03.972624 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:03.972627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:03.972629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:03.972715 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:03.973110 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:03.973115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:03.973116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:03.973388 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:03.973512 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:03.974172 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 22:24:03.974316 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:24:03.975368 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:03.961862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:42983)
0811 22:24:03.962366 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:03.962413 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:03.963747 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:03.964283 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:03.964291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:03.964692 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:03.965270 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:03.965287 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.966094 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.966097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:03.967204 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:03.967214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:03.967779 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:03.967782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:03.967933 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:03.968515 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:03.968532 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:03.970160 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:03.972172 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:03.972178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:03.972194 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:03.972501 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:03.972803 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:03.972807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:03.972809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:03.973005 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:03.973512 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:03.973520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:03.973800 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 22:24:03.974018 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 22:24:03.974351 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:24:03.976919 19010 heartbeater.cc:347] Connected to a master server at 127.26.125.62:36329
I20260811 22:24:03.977197 19010 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:03.977770 19010 heartbeater.cc:511] Master 127.26.125.62:36329 requested a full tablet report, sending...
I20260811 22:24:03.979079 18519 ts_manager.cc:194] Registered new tserver with Master: 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:24:03.979923 18519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:42983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:24:03.997615 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:36329
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:04.127696 19018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:04.128031 19018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:04.128096 19018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:04.132310 19018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:04.132468 19018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:04.132562 19018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:24:04.138132 19018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:04.139351 19018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:04.140525 19018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:04.150940 19028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:04.150929 19026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:04.150925 19024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:04.151376 19018 server_base.cc:1034] running on GCE node
I20260811 22:24:04.151829 19018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:04.152551 19018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:04.153726 19018 hybrid_clock.cc:648] HybridClock initialized: now 1786487044153692 us; error 49 us; skew 500 ppm
Aug 11 22:24:04 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487044, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:04.156869 19018 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:24:04.157869 19018 webserver.cc:460] Webserver started at http://127.26.125.3:39945/ using document root <none> and password file <none>
I20260811 22:24:04.158484 19018 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:04.158560 19018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:04.158782 19018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:04.161038 19018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a"
format_stamp: "Formatted at 2026-08-11 22:24:04 on dist-test-slave-5bn2"
server_key: "adbe2ec992a9fd6d0ea65f7082aa10a6"
server_key_iv: "87707b1e085407826417517bf3e5d5b9"
server_key_version: "encryptionkey@0"
I20260811 22:24:04.161628 19018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a"
format_stamp: "Formatted at 2026-08-11 22:24:04 on dist-test-slave-5bn2"
server_key: "adbe2ec992a9fd6d0ea65f7082aa10a6"
server_key_iv: "87707b1e085407826417517bf3e5d5b9"
server_key_version: "encryptionkey@0"
I20260811 22:24:04.165699 19018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:24:04.168541 19036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:04.169867 19018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:24:04.170001 19018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a"
format_stamp: "Formatted at 2026-08-11 22:24:04 on dist-test-slave-5bn2"
server_key: "adbe2ec992a9fd6d0ea65f7082aa10a6"
server_key_iv: "87707b1e085407826417517bf3e5d5b9"
server_key_version: "encryptionkey@0"
I20260811 22:24:04.170133 19018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:04.193023 19018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:04.196471 19018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:04.196671 19018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:04.197307 19018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:04.198201 19018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:04.198251 19018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:04.198307 19018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:04.198333 19018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:04.207257 19018 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:34611
I20260811 22:24:04.207302 19225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:34611 every 8 connection(s)
I20260811 22:24:04.208230 19018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:24:04.216281 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 19018
I20260811 22:24:04.216426 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:24:04.216718 27124 external_mini_cluster.cc:1428] Setting key 879404e3b883d747248c755aa8803a8c
Aug 11 22:24:04 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487044, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:04.222123 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.210146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:45811 (local address 127.26.125.62:36329)
0811 22:24:04.210308 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:04.210313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:04.211251 (+ 938us) server_negotiation.cc:407] Connection header received
0811 22:24:04.212563 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:04.212566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:04.212616 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:04.212689 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:04.213870 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.214410 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.215549 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.215765 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.215841 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.218566 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:04.218569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:04.218571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:04.218596 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:04.220048 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:04.220566 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:04.220570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:04.220571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:04.220651 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:04.220987 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:04.220991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:04.220992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:04.221185 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:04.221272 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:04.221813 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 22:24:04.221943 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:24:04.223053 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.210557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:45811)
0811 22:24:04.211084 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:04.211131 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:04.212360 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:04.212846 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:04.212853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:04.213226 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:04.213731 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.213746 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.214498 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.214500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.215443 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.215453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.215881 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.215884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.216032 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.216643 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:04.216660 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:04.218393 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:04.220165 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:04.220172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:04.220190 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:04.220470 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:04.220735 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:04.220738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:04.220740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:04.220893 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:04.221288 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:04.221295 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:04.221575 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 22:24:04.221753 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 22:24:04.222118 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":122,"threads_started":1}
I20260811 22:24:04.224263 19226 heartbeater.cc:347] Connected to a master server at 127.26.125.62:36329
I20260811 22:24:04.224514 19226 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:04.225032 19226 heartbeater.cc:511] Master 127.26.125.62:36329 requested a full tablet report, sending...
I20260811 22:24:04.226063 18519 ts_manager.cc:194] Registered new tserver with Master: b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611)
I20260811 22:24:04.226619 18519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:45811
I20260811 22:24:04.230660 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:24:04.239532 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.232627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.26.125.62:36329)
0811 22:24:04.232754 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:04.232758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:04.232981 (+ 223us) server_negotiation.cc:407] Connection header received
0811 22:24:04.233051 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:04.233053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:04.233092 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:04.233152 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:04.233763 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.234275 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.235137 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.235346 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.235419 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.236650 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:04.236653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:04.236655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:04.236678 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:04.237946 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:04.238369 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:04.238372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:04.238373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:04.238452 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:04.238678 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:04.238681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:04.238682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:04.238942 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:04.239063 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:04.239282 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 22:24:04.239381 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:24:04.242910 18519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:24:04.246055 18519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:24:04.261492 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.255708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46629 (local address 127.0.0.1:35870)
0811 22:24:04.256458 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:04.256472 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:04.256551 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:04.257393 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:04.257396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:04.257415 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:04.257513 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.257519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.258787 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.258790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.259919 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.259932 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.261038 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.261042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.261160 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.261188 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:24:04.261241 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:24:04.261316 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260811 22:24:04.261523 19250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.255058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.26.125.3:34611)
0811 22:24:04.256290 (+ 1232us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:04.256304 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:04.256324 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:24:04.256388 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:04.256392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:04.256549 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:04.256680 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:04.257012 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.257893 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.259609 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.260586 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.260984 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.261144 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:04.261198 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 22:24:04.261295 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1119,"thread_start_us":152,"threads_started":1}
I20260811 22:24:04.261655 19252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.255218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:34653 (local address 127.0.0.1:32914)
0811 22:24:04.255777 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:04.255838 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:04.255981 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:04.256629 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:04.256636 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:04.256659 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:04.256891 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.256899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.258306 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.258309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.259469 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.259476 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.260867 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.260871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.261378 (+ 507us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.261399 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:24:04.261465 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:24:04.261534 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":80,"spinlock_wait_cycles":3712,"thread_start_us":98,"threads_started":1}
I20260811 22:24:04.262022 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.254892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:34611 (local address 127.0.0.1:35920)
0811 22:24:04.255777 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:04.255837 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:04.255981 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:04.256655 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:04.256657 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:04.256669 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:04.256889 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.256896 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.258008 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.258011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.259452 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.259472 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.260759 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.260763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.260932 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.261024 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 22:24:04.261143 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:24:04.261288 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":91,"spinlock_wait_cycles":7168,"thread_start_us":100,"threads_started":1}
I20260811 22:24:04.262169 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.255386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.26.125.2:34653)
0811 22:24:04.255819 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:04.255832 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:04.256125 (+ 293us) server_negotiation.cc:407] Connection header received
0811 22:24:04.256210 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:04.256217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:04.256450 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:04.256622 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:04.257027 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.258163 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.259609 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.260515 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.260625 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.260809 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:04.261757 (+ 948us) server_negotiation.cc:299] Negotiation successful
0811 22:24:04.262008 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":139,"threads_started":1}
I20260811 22:24:04.263492 19257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.256094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.26.125.1:46629)
0811 22:24:04.256939 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:04.256946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:04.256967 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:24:04.257041 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:04.257046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:04.257249 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:04.257397 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:04.257625 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.258666 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.260096 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.260897 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.263001 (+ 2104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.263191 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:04.263248 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:24:04.263350 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":97,"threads_started":1}
I20260811 22:24:04.265420 19122 tablet_service.cc:1467] Processing CreateTablet for tablet ac13dec771e44b45a4937983fa4d9bf2 (DEFAULT_TABLE table=test-table [id=d75fe95ef6d24249b96038bf3039c70f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:04.266000 18702 tablet_service.cc:1467] Processing CreateTablet for tablet ac13dec771e44b45a4937983fa4d9bf2 (DEFAULT_TABLE table=test-table [id=d75fe95ef6d24249b96038bf3039c70f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:04.266495 19122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac13dec771e44b45a4937983fa4d9bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:04.267151 18702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac13dec771e44b45a4937983fa4d9bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:04.268160 18910 tablet_service.cc:1467] Processing CreateTablet for tablet ac13dec771e44b45a4937983fa4d9bf2 (DEFAULT_TABLE table=test-table [id=d75fe95ef6d24249b96038bf3039c70f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:04.269270 18910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac13dec771e44b45a4937983fa4d9bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:04.273172 19259 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Bootstrap starting.
I20260811 22:24:04.275280 19260 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Bootstrap starting.
I20260811 22:24:04.276665 19259 tablet_bootstrap.cc:654] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:04.277566 19261 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Bootstrap starting.
I20260811 22:24:04.277645 19260 tablet_bootstrap.cc:654] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:04.277768 19259 log.cc:824] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:04.278476 19260 log.cc:824] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:04.280194 19260 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: No bootstrap required, opened a new log
I20260811 22:24:04.280198 19259 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: No bootstrap required, opened a new log
I20260811 22:24:04.280326 19261 tablet_bootstrap.cc:654] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:04.280406 19259 ts_tablet_manager.cc:1397] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:24:04.280405 19260 ts_tablet_manager.cc:1397] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:24:04.281219 19261 log.cc:824] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:04.283555 19261 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: No bootstrap required, opened a new log
I20260811 22:24:04.283783 19261 ts_tablet_manager.cc:1397] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:24:04.284075 19259 raft_consensus.cc:348] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.284327 19259 raft_consensus.cc:374] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:04.284391 19259 raft_consensus.cc:729] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5c31bbd7a7e4709ab44fb091d28cf4a, State: Initialized, Role: FOLLOWER
I20260811 22:24:04.284670 19260 raft_consensus.cc:348] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.284844 19259 consensus_queue.cc:260] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.284945 19260 raft_consensus.cc:374] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:04.285004 19260 raft_consensus.cc:729] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c1f08b3435460ea2ea7b91ad846a78, State: Initialized, Role: FOLLOWER
I20260811 22:24:04.285523 19226 heartbeater.cc:503] Master 127.26.125.62:36329 was elected leader, sending a full tablet report...
I20260811 22:24:04.285518 19260 consensus_queue.cc:260] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.286602 18795 heartbeater.cc:503] Master 127.26.125.62:36329 was elected leader, sending a full tablet report...
I20260811 22:24:04.286615 19260 ts_tablet_manager.cc:1428] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:24:04.287554 19261 raft_consensus.cc:348] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.287827 19261 raft_consensus.cc:374] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:04.287882 19261 raft_consensus.cc:729] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f9ae43d7974caca32cf1b2608cfbb5, State: Initialized, Role: FOLLOWER
I20260811 22:24:04.288079 19259 ts_tablet_manager.cc:1428] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:24:04.288345 19261 consensus_queue.cc:260] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.289098 19010 heartbeater.cc:503] Master 127.26.125.62:36329 was elected leader, sending a full tablet report...
I20260811 22:24:04.289475 19261 ts_tablet_manager.cc:1428] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 22:24:04.404241 19267 raft_consensus.cc:482] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:04.404472 19267 raft_consensus.cc:504] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.405757 19267 leader_election.cc:288] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629), 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
I20260811 22:24:04.409816 19257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.406200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:52913 (local address 127.26.125.1:46629)
0811 22:24:04.406320 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:04.406325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:04.406634 (+ 309us) server_negotiation.cc:407] Connection header received
0811 22:24:04.406751 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:04.406755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:04.406797 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:04.406897 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:04.407179 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.407712 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.408778 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.409209 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.409311 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.409364 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:04.409599 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:24:04.409685 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:24:04.409807 19273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.406046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46629 (local address 127.26.125.3:52913)
0811 22:24:04.406520 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:04.406531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:04.406657 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:04.406902 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:04.406905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:04.406920 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:04.407038 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.407045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.407831 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.407834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.408648 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.408659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.409346 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.409350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.409457 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.409486 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:24:04.409567 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:24:04.409635 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":93,"thread_start_us":156,"threads_started":1}
I20260811 22:24:04.409862 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.406165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:49925 (local address 127.26.125.2:34653)
0811 22:24:04.406302 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:04.406307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:04.406416 (+ 109us) server_negotiation.cc:407] Connection header received
0811 22:24:04.406574 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:04.406577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:04.406626 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:04.406692 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:04.407008 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.407573 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.408729 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.409262 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.409459 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.409510 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:04.409678 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:24:04.409762 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:24:04.409818 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.406046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:34653 (local address 127.26.125.3:49925)
0811 22:24:04.406314 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:04.406330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:04.406455 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:04.406720 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:04.406723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:04.406740 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:04.406877 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.406883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.407680 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.407683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.408588 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:04.408599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.409398 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.409402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:04.409548 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.409574 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:24:04.409651 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:24:04.409715 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":43}
I20260811 22:24:04.410620 18731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac13dec771e44b45a4937983fa4d9bf2" candidate_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" is_pre_election: true
I20260811 22:24:04.410895 18937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac13dec771e44b45a4937983fa4d9bf2" candidate_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" is_pre_election: true
I20260811 22:24:04.410985 18731 raft_consensus.cc:2454] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5c31bbd7a7e4709ab44fb091d28cf4a in term 0.
I20260811 22:24:04.411247 18937 raft_consensus.cc:2454] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5c31bbd7a7e4709ab44fb091d28cf4a in term 0.
I20260811 22:24:04.411530 19043 leader_election.cc:302] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5c31bbd7a7e4709ab44fb091d28cf4a, f2c1f08b3435460ea2ea7b91ad846a78; no voters:
I20260811 22:24:04.411846 19267 raft_consensus.cc:2781] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:24:04.411942 19267 raft_consensus.cc:482] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:24:04.411990 19267 raft_consensus.cc:3037] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:04.413125 19267 raft_consensus.cc:504] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.413463 19267 leader_election.cc:288] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [CANDIDATE]: Term 1 election: Requested vote from peers f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629), 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
I20260811 22:24:04.413839 18937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac13dec771e44b45a4937983fa4d9bf2" candidate_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f9ae43d7974caca32cf1b2608cfbb5"
I20260811 22:24:04.413954 18937 raft_consensus.cc:3037] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:04.415028 18937 raft_consensus.cc:2454] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5c31bbd7a7e4709ab44fb091d28cf4a in term 1.
I20260811 22:24:04.415349 19042 leader_election.cc:302] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44f9ae43d7974caca32cf1b2608cfbb5, b5c31bbd7a7e4709ab44fb091d28cf4a; no voters:
I20260811 22:24:04.415331 18731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac13dec771e44b45a4937983fa4d9bf2" candidate_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c1f08b3435460ea2ea7b91ad846a78"
I20260811 22:24:04.415453 18731 raft_consensus.cc:3037] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:04.415541 19267 raft_consensus.cc:2781] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:04.415762 19267 raft_consensus.cc:686] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 LEADER]: Becoming Leader. State: Replica: b5c31bbd7a7e4709ab44fb091d28cf4a, State: Running, Role: LEADER
I20260811 22:24:04.416098 19267 consensus_queue.cc:237] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:04.416751 18731 raft_consensus.cc:2454] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5c31bbd7a7e4709ab44fb091d28cf4a in term 1.
I20260811 22:24:04.419327 18519 catalog_manager.cc:5495] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a reported cstate change: term changed from 0 to 1, leader changed from <none> to b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3). New cstate: current_term: 1 leader_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } health_report { overall_health: UNKNOWN } } }
W20260811 22:24:04.449410 18796 tablet.cc:2369] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:04.457717 19250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.454106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.26.125.3:34611)
0811 22:24:04.454242 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:04.454246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:04.454261 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:24:04.454352 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:04.454355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:04.454412 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:04.454510 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:04.454772 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.455342 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.456224 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:04.456442 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:04.456537 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:04.456852 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:04.457301 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:04.457413 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:04.457487 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 22:24:04.457549 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 22:24:04.459131 19227 tablet.cc:2369] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:24:04.460665 19011 tablet.cc:2369] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:04.463946 18937 raft_consensus.cc:1261] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 1 FOLLOWER]: Refusing update from remote peer b5c31bbd7a7e4709ab44fb091d28cf4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:04.463948 18731 raft_consensus.cc:1261] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Refusing update from remote peer b5c31bbd7a7e4709ab44fb091d28cf4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:04.464640 19267 consensus_queue.cc:1035] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:04.464841 19274 consensus_queue.cc:1035] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:04.473976 19283 mvcc.cc:205] Tried to move back new op lower bound from 7317450934117015552 to 7317450933931196416. Current Snapshot: MvccSnapshot[applied={T|T < 7317450934117015552}]
I20260811 22:24:04.474550 19282 mvcc.cc:205] Tried to move back new op lower bound from 7317450934117015552 to 7317450933931196416. Current Snapshot: MvccSnapshot[applied={T|T < 7317450934117015552}]
I20260811 22:24:04.481750 18519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:24:04.481977 18519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:24:04.484683 18519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Sending DeleteTablet for 3 replicas of tablet ac13dec771e44b45a4937983fa4d9bf2
I20260811 22:24:04.485556 19122 tablet_service.cc:1514] Processing DeleteTablet for tablet ac13dec771e44b45a4937983fa4d9bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:04 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35920
I20260811 22:24:04.485579 18910 tablet_service.cc:1514] Processing DeleteTablet for tablet ac13dec771e44b45a4937983fa4d9bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:04 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:32914
I20260811 22:24:04.485656 18702 tablet_service.cc:1514] Processing DeleteTablet for tablet ac13dec771e44b45a4937983fa4d9bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:04 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35870
I20260811 22:24:04.485792 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 18607
I20260811 22:24:04.505333 19294 tablet_replica.cc:330] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: stopping tablet replica
I20260811 22:24:04.505760 19294 raft_consensus.cc:2229] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:24:04.506119 19294 raft_consensus.cc:2258] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:04.511189 19294 ts_tablet_manager.cc:1905] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:04.511566 19295 tablet_replica.cc:330] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: stopping tablet replica
I20260811 22:24:04.512033 19295 raft_consensus.cc:2229] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:24:04.512663 19295 raft_consensus.cc:2258] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:04.514315 19294 ts_tablet_manager.cc:1918] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:04.514420 19294 log.cc:1196] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/ac13dec771e44b45a4937983fa4d9bf2
I20260811 22:24:04.514665 19294 ts_tablet_manager.cc:1939] T ac13dec771e44b45a4937983fa4d9bf2 P 44f9ae43d7974caca32cf1b2608cfbb5: Deleting consensus metadata
I20260811 22:24:04.517309 19295 ts_tablet_manager.cc:1905] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:04.520085 19295 ts_tablet_manager.cc:1918] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:04.520197 19295 log.cc:1196] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/ac13dec771e44b45a4937983fa4d9bf2
I20260811 22:24:04.520473 19295 ts_tablet_manager.cc:1939] T ac13dec771e44b45a4937983fa4d9bf2 P b5c31bbd7a7e4709ab44fb091d28cf4a: Deleting consensus metadata
I20260811 22:24:04.521766 18490 catalog_manager.cc:4841] TS b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611): tablet ac13dec771e44b45a4937983fa4d9bf2 (table test-table [id=d75fe95ef6d24249b96038bf3039c70f]) successfully deleted
I20260811 22:24:04.530192 18489 catalog_manager.cc:4841] TS 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653): tablet ac13dec771e44b45a4937983fa4d9bf2 (table test-table [id=d75fe95ef6d24249b96038bf3039c70f]) successfully deleted
W20260811 22:24:04.542948 18490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:46629 (error 108)
I20260811 22:24:04.544799 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:04.544399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46629 (local address 127.0.0.1:35874)
0811 22:24:04.544590 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:04.544682 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:24:04.545033 18490 catalog_manager.cc:4584] TS f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ac13dec771e44b45a4937983fa4d9bf2: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46629: connect: Connection refused (error 111)
I20260811 22:24:04.549405 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 18802
I20260811 22:24:04.575642 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 19018
I20260811 22:24:04.591923 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 18457
I20260811 22:24:04.610141 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36329
--webserver_interface=127.26.125.62
--webserver_port=41575
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:04.728108 19299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:04.728435 19299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:04.728480 19299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:04.732821 19299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:04.732904 19299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:04.732923 19299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:04.732940 19299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:04.732954 19299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:04.738610 19299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=41575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:04.739847 19299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:04.740887 19299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:04.757426 19314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:04.757447 19312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:04.757453 19309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:04.757498 19299 server_base.cc:1034] running on GCE node
I20260811 22:24:04.758461 19299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:04.759753 19299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:04.760921 19299 hybrid_clock.cc:648] HybridClock initialized: now 1786487044760900 us; error 37 us; skew 500 ppm
Aug 11 22:24:04 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487044, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:04.763944 19299 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:24:04.764966 19299 webserver.cc:460] Webserver started at http://127.26.125.62:41575/ using document root <none> and password file <none>
I20260811 22:24:04.765581 19299 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:04.765673 19299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:04.769968 19299 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:04.772444 19324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:04.773828 19299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:24:04.773972 19299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "d4d59c48f5d24a46b2aa332d90de4eed"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "4e90f89d3d4ee61d5f9a0a59fccc09d7"
server_key_iv: "f37e47d0ab01f700659d4dab21085641"
server_key_version: "encryptionkey@0"
I20260811 22:24:04.774621 19299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:04.792227 19299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:04.795626 19299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:04.795832 19299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:04.803423 19299 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:36329
I20260811 22:24:04.803700 19409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:36329 every 8 connection(s)
I20260811 22:24:04.804677 19299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:04.807988 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 19299
I20260811 22:24:04.808527 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:46629
--local_ip_for_outbound_sockets=127.26.125.1
--tserver_master_addrs=127.26.125.62:36329
--webserver_port=40905
--webserver_interface=127.26.125.1
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:04.810022 19411 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:24:04.812050 19411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Bootstrap starting.
I20260811 22:24:04.829241 19411 log.cc:824] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:04.838783 19411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:24:04.839232 19411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Bootstrap complete.
I20260811 22:24:04.843293 19411 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:04.843720 19411 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d59c48f5d24a46b2aa332d90de4eed, State: Initialized, Role: FOLLOWER
I20260811 22:24:04.844254 19411 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:04.844391 19411 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:04.844460 19411 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:04.844560 19411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:24:04.846582 19411 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:04.846978 19411 leader_election.cc:302] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4d59c48f5d24a46b2aa332d90de4eed; no voters:
I20260811 22:24:04.847311 19411 leader_election.cc:288] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:24:04.847425 19420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:24:04.847630 19420 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 2 LEADER]: Becoming Leader. State: Replica: d4d59c48f5d24a46b2aa332d90de4eed, State: Running, Role: LEADER
I20260811 22:24:04.847941 19420 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:04.848659 19411 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:04.848999 19420 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4d59c48f5d24a46b2aa332d90de4eed. Latest consensus state: current_term: 2 leader_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } } }
I20260811 22:24:04.849223 19420 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:04.849851 19422 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } } }
I20260811 22:24:04.850025 19422 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:04.850353 19427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:04.855603 19427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:04.856424 19427 catalog_manager.cc:1259] Loaded cluster ID: 6a479ffbbd6246bd94658ddd0c8e5c8e
I20260811 22:24:04.856494 19427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:04.857961 19427 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 22:24:04.862389 19427 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 22:24:04.868397 19427 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:04.869323 19427 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Loaded TSK: 0
I20260811 22:24:04.870642 19427 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:24:04.934429 19416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:04.934701 19416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:04.934743 19416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:04.938822 19416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:04.938930 19416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:04.938998 19416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:24:04.944854 19416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:46629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=40905
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:04.945987 19416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:04.947141 19416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:04.957204 19462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:04.957185 19461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:04.957243 19465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:04.957885 19416 server_base.cc:1034] running on GCE node
I20260811 22:24:04.958382 19416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:04.959245 19416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:04.960457 19416 hybrid_clock.cc:648] HybridClock initialized: now 1786487044960426 us; error 40 us; skew 500 ppm
Aug 11 22:24:04 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487044, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:04.964078 19416 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:24:04.965056 19416 webserver.cc:460] Webserver started at http://127.26.125.1:40905/ using document root <none> and password file <none>
I20260811 22:24:04.965651 19416 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:04.965734 19416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:04.970001 19416 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:04.972491 19477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:04.973989 19416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:24:04.974128 19416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "f2c1f08b3435460ea2ea7b91ad846a78"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "d9820c319a832f1f4d7988cb702c990e"
server_key_iv: "50a06d4ebd22295bbe49587effdf7f80"
server_key_version: "encryptionkey@0"
I20260811 22:24:04.974762 19416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:04.992007 19416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:04.995602 19416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:04.995800 19416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:04.996477 19416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:04.997678 19485 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:24:05.002266 19416 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:24:05.002357 19416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:24:05.002435 19416 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:24:05.004293 19416 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:24:05.004348 19416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 22:24:05.004686 19485 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Bootstrap starting.
I20260811 22:24:05.014048 19416 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:46629
I20260811 22:24:05.014096 19659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:46629 every 8 connection(s)
I20260811 22:24:05.015131 19416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:24:05.017807 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 19416
I20260811 22:24:05.018357 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:34653
--local_ip_for_outbound_sockets=127.26.125.2
--tserver_master_addrs=127.26.125.62:36329
--webserver_port=39327
--webserver_interface=127.26.125.2
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:05.024701 19485 log.cc:824] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:24:05 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487044, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:05.032980 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.017181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:41261)
0811 22:24:05.017863 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.017922 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:05.019127 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:05.020803 (+ 1676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:05.020810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:05.021188 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:05.021663 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.021677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.023128 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.023131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.024085 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.024096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.024896 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.024901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.025104 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.025881 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:05.025900 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:05.027412 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:05.029569 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:05.029576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:05.029592 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:05.029866 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:05.030247 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:05.030250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:05.030252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:05.030399 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:05.031088 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:05.031094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:05.031370 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 22:24:05.031544 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 22:24:05.031988 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":136,"threads_started":1}
I20260811 22:24:05.032966 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.017278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:41261 (local address 127.26.125.62:36329)
0811 22:24:05.018422 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.018571 (+ 149us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.018624 (+ 53us) server_negotiation.cc:407] Connection header received
0811 22:24:05.019682 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.019695 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.020074 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.020403 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:05.021849 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.022957 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.024216 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.024722 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.024953 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.027625 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:05.027631 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:05.027647 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:05.027701 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:05.029434 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.029987 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.029992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.029997 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.030151 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.030492 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.030495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.030496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.030972 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:05.031110 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.031698 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.031988 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1000,"thread_start_us":131,"threads_started":1}
I20260811 22:24:05.034921 19485 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:24:05.035375 19485 tablet_bootstrap.cc:492] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Bootstrap complete.
I20260811 22:24:05.035632 19660 heartbeater.cc:347] Connected to a master server at 127.26.125.62:36329
I20260811 22:24:05.035867 19660 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:05.036369 19660 heartbeater.cc:511] Master 127.26.125.62:36329 requested a full tablet report, sending...
I20260811 22:24:05.036778 19485 ts_tablet_manager.cc:1397] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Time spent bootstrapping tablet: real 0.032s user 0.030s sys 0.000s
I20260811 22:24:05.039110 19347 ts_manager.cc:194] Registered new tserver with Master: f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629)
I20260811 22:24:05.039819 19485 raft_consensus.cc:348] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:05.040359 19485 raft_consensus.cc:729] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c1f08b3435460ea2ea7b91ad846a78, State: Initialized, Role: FOLLOWER
I20260811 22:24:05.040895 19485 consensus_queue.cc:260] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } }
I20260811 22:24:05.042023 19485 ts_tablet_manager.cc:1428] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:24:05.043344 19347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:41261
I20260811 22:24:05.046249 19660 heartbeater.cc:503] Master 127.26.125.62:36329 was elected leader, sending a full tablet report...
Aug 11 22:24:05 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487044, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:24:05.053956 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.042306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.26.125.1:46629)
0811 22:24:05.042605 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.042617 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.043230 (+ 613us) server_negotiation.cc:407] Connection header received
0811 22:24:05.043698 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.043705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.043927 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.044076 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:05.044841 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.045625 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.046579 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.047108 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.047217 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.050211 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:05.050220 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:05.050234 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:05.050260 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:05.051884 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.052438 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.052443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.052444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.052528 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.052789 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.052791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.052792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.053052 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:05.053159 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.053659 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.053814 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
I20260811 22:24:05.053963 19672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.042627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46629 (local address 127.0.0.1:35886)
0811 22:24:05.043084 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.043137 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:05.043348 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:05.044101 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:05.044108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:05.044600 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:05.044713 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.044720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.045771 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.045774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.046458 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.046469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.047231 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.047234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.047344 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.047998 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:05.048013 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:05.049963 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:05.051998 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:05.052010 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:05.052016 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:05.052332 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:05.052607 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:05.052609 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:05.052611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:05.052708 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:05.053245 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:05.053252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:05.053398 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 22:24:05.053556 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 22:24:05.053778 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
I20260811 22:24:05.055078 19545 tablet_service.cc:1514] Processing DeleteTablet for tablet ac13dec771e44b45a4937983fa4d9bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:04 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35886
I20260811 22:24:05.056036 19673 tablet_replica.cc:330] stopping tablet replica
I20260811 22:24:05.056344 19673 raft_consensus.cc:2229] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:24:05.056591 19673 raft_consensus.cc:2258] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:05.057602 19673 ts_tablet_manager.cc:1905] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:05.061650 19673 ts_tablet_manager.cc:1918] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:05.061750 19673 log.cc:1196] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/ac13dec771e44b45a4937983fa4d9bf2
I20260811 22:24:05.061990 19673 ts_tablet_manager.cc:1939] T ac13dec771e44b45a4937983fa4d9bf2 P f2c1f08b3435460ea2ea7b91ad846a78: Deleting consensus metadata
I20260811 22:24:05.063877 19327 catalog_manager.cc:4841] TS f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629): tablet ac13dec771e44b45a4937983fa4d9bf2 (table test-table [id=d75fe95ef6d24249b96038bf3039c70f]) successfully deleted
W20260811 22:24:05.148510 19666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:05.148811 19666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:05.148874 19666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:05.153131 19666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:05.153221 19666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:05.153309 19666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:24:05.159065 19666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:34653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=39327
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:05.160245 19666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:05.161340 19666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:05.171481 19680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:05.171566 19682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:05.171486 19687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:05.171558 19666 server_base.cc:1034] running on GCE node
I20260811 22:24:05.172199 19666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:05.172830 19666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:05.173996 19666 hybrid_clock.cc:648] HybridClock initialized: now 1786487045173956 us; error 49 us; skew 500 ppm
Aug 11 22:24:05 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487045, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:05.176717 19666 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:24:05.177644 19666 webserver.cc:460] Webserver started at http://127.26.125.2:39327/ using document root <none> and password file <none>
I20260811 22:24:05.178267 19666 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:05.178357 19666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:05.182507 19666 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:05.184737 19700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:05.186002 19666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:24:05.186117 19666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "44f9ae43d7974caca32cf1b2608cfbb5"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "cce361a0de9f2f647c58489b2f140783"
server_key_iv: "54b42fd8f8e6f13213d901e63cf5c160"
server_key_version: "encryptionkey@0"
I20260811 22:24:05.186671 19666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:05.233783 19666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:05.236876 19666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:05.237092 19666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:05.237735 19666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:05.238680 19666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:05.238734 19666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:05.238794 19666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:05.238824 19666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:05.247713 19666 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:34653
I20260811 22:24:05.247735 19876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:34653 every 8 connection(s)
I20260811 22:24:05.248863 19666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:24:05.249117 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 19666
I20260811 22:24:05.249648 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:34611
--local_ip_for_outbound_sockets=127.26.125.3
--tserver_master_addrs=127.26.125.62:36329
--webserver_port=39945
--webserver_interface=127.26.125.3
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:24:05 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487045, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:05.266914 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.251173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:46473 (local address 127.26.125.62:36329)
0811 22:24:05.251331 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.251336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.252410 (+ 1074us) server_negotiation.cc:407] Connection header received
0811 22:24:05.254184 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.254187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.254243 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.254324 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:05.256188 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.256784 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.258559 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.258810 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.258944 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.262121 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:05.262125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:05.262127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:05.262157 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:05.263872 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.264633 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.264638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.264640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.264751 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.265246 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.265250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.265252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.265505 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:05.265605 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.266461 (+ 856us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.266676 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:24:05.267831 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.251648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:46473)
0811 22:24:05.252212 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.252275 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:05.253905 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:05.254559 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:05.254577 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:05.255197 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:05.255994 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.256020 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.256929 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.256933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.258406 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.258419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.259008 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.259012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.259239 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.259920 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:05.259946 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:05.261875 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:05.264028 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:05.264035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:05.264055 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:05.264443 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:05.264888 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:05.264894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:05.264896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:05.265144 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:05.265668 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:05.265676 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:05.266092 (+ 416us) client_negotiation.cc:769] Sending connection context
0811 22:24:05.266334 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 22:24:05.266775 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":135,"threads_started":1}
I20260811 22:24:05.269181 19877 heartbeater.cc:347] Connected to a master server at 127.26.125.62:36329
I20260811 22:24:05.269459 19877 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:05.270033 19877 heartbeater.cc:511] Master 127.26.125.62:36329 requested a full tablet report, sending...
I20260811 22:24:05.271256 19347 ts_manager.cc:194] Registered new tserver with Master: 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
I20260811 22:24:05.272033 19347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:46473
W20260811 22:24:05.372452 19879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:05.372733 19879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:05.372777 19879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:05.376793 19879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:05.376863 19879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:05.376922 19879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:24:05.382371 19879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:34611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=39945
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:05.383484 19879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:05.384501 19879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:05.395287 19894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:05.395278 19890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:05.395278 19892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:05.395596 19879 server_base.cc:1034] running on GCE node
I20260811 22:24:05.396106 19879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:05.396842 19879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:05.398005 19879 hybrid_clock.cc:648] HybridClock initialized: now 1786487045397975 us; error 45 us; skew 500 ppm
Aug 11 22:24:05 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487045, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:05.400897 19879 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:24:05.401849 19879 webserver.cc:460] Webserver started at http://127.26.125.3:39945/ using document root <none> and password file <none>
I20260811 22:24:05.402427 19879 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:05.402518 19879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:05.406694 19879 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:05.408977 19903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:05.410302 19879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:24:05.410421 19879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a"
format_stamp: "Formatted at 2026-08-11 22:24:04 on dist-test-slave-5bn2"
server_key: "adbe2ec992a9fd6d0ea65f7082aa10a6"
server_key_iv: "87707b1e085407826417517bf3e5d5b9"
server_key_version: "encryptionkey@0"
I20260811 22:24:05.411052 19879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:05.427023 19879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:05.430469 19879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:05.430693 19879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:05.431401 19879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:05.432312 19879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:05.432371 19879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:05.432431 19879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:05.432457 19879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:05.441184 19879 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:34611
I20260811 22:24:05.441219 20091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:34611 every 8 connection(s)
I20260811 22:24:05.442253 19879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:24:05.449257 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 19879
Aug 11 22:24:05 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487045, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:05.457355 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.444177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:33213 (local address 127.26.125.62:36329)
0811 22:24:05.444612 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.444619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.445443 (+ 824us) server_negotiation.cc:407] Connection header received
0811 22:24:05.446893 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.446896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.446958 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.447038 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:05.448345 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.448874 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.450618 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.450950 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.451057 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.453709 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:05.453712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:05.453714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:05.453734 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:05.455126 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.455684 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.455688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.455690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.455798 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.456175 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.456179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.456181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.456412 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:05.456508 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.457074 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.457177 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 22:24:05.458521 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.450050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.26.125.62:36329)
0811 22:24:05.450367 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.450371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.450467 (+ 96us) server_negotiation.cc:407] Connection header received
0811 22:24:05.450579 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.450582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.450638 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.450710 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:05.451593 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.452325 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.453200 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.453422 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.453499 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.455226 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:05.455229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:05.455232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:05.455256 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:05.456771 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.457211 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.457214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.457216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.457296 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:05.457611 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:05.457615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:05.457617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:05.457835 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:05.458043 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.458196 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.458315 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":102,"threads_started":1}
I20260811 22:24:05.458520 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.444736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:33213)
0811 22:24:05.445265 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.445314 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:05.446635 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:05.447256 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:05.447264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:05.447670 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:05.448188 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.448203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.449071 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.449075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.450445 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.450503 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.451110 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.451113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.451268 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.451865 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:05.451887 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:05.453534 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:05.455242 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:05.455249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:05.455264 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:05.455579 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:05.455900 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:05.455902 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:05.455904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:05.456078 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:05.456530 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:05.456536 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:05.456796 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 22:24:05.457028 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 22:24:05.457376 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":137,"threads_started":1}
I20260811 22:24:05.460270 20092 heartbeater.cc:347] Connected to a master server at 127.26.125.62:36329
I20260811 22:24:05.460531 20092 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:05.461021 20092 heartbeater.cc:511] Master 127.26.125.62:36329 requested a full tablet report, sending...
I20260811 22:24:05.462039 19346 ts_manager.cc:194] Registered new tserver with Master: b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611)
I20260811 22:24:05.462611 19346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:33213
I20260811 22:24:05.472437 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:24:05.476768 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.473056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60758 (local address 127.26.125.62:36329)
0811 22:24:05.473181 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.473184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.473381 (+ 197us) server_negotiation.cc:407] Connection header received
0811 22:24:05.473581 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.473584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.473697 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.473784 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:05.473994 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.474549 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.475401 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.475667 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.475752 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.475986 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:05.476408 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:05.476547 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.476579 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.476636 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:24:05.477481 19346 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:24:05.481055 19346 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:24:05.489573 19545 tablet_service.cc:1467] Processing CreateTablet for tablet 96baa3ef8cd74efeb520782f900be57e (DEFAULT_TABLE table=test-table [id=bcd955bf37d845eaa7890f2ea047e72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:05.490337 19545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96baa3ef8cd74efeb520782f900be57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:05.493184 19485 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Bootstrap starting.
I20260811 22:24:05.494091 19485 tablet_bootstrap.cc:654] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:05.494704 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.488603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:34653 (local address 127.0.0.1:32928)
0811 22:24:05.488945 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.488957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:05.489038 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:05.489887 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:05.489889 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:05.489927 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:05.490118 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.490127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.492023 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.492026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.493109 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.493118 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.494234 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.494238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.494329 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.494382 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:24:05.494457 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:24:05.494542 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":38,"spinlock_wait_cycles":3712,"thread_start_us":156,"threads_started":1}
I20260811 22:24:05.494905 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.488599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.26.125.3:34611)
0811 22:24:05.489262 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.489272 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.489294 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:24:05.489380 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.489386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.489660 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.490285 (+ 625us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:05.490602 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.491610 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.493142 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.494228 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.494326 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.494494 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.494627 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.494728 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":172,"threads_started":1}
I20260811 22:24:05.495252 19672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.488403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:34611 (local address 127.0.0.1:35942)
0811 22:24:05.488589 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.488602 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:05.488671 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:05.489883 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:05.489886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:05.489926 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:05.490072 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.490093 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.491742 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.491745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.492976 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.492987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.494340 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.494344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.494467 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.494499 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:24:05.494584 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:24:05.494665 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":4,"spinlock_wait_cycles":26240}
I20260811 22:24:05.495349 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.489110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32928 (local address 127.26.125.2:34653)
0811 22:24:05.489487 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.489497 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.489517 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:24:05.489586 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.489590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.489744 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.490013 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:05.490274 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.491866 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.493232 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.494093 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.494947 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.495100 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.495148 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.495219 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":162,"threads_started":1}
I20260811 22:24:05.496062 19485 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: No bootstrap required, opened a new log
I20260811 22:24:05.496162 19485 ts_tablet_manager.cc:1397] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:05.496627 19485 raft_consensus.cc:348] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.496745 19485 raft_consensus.cc:374] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:05.496781 19485 raft_consensus.cc:729] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c1f08b3435460ea2ea7b91ad846a78, State: Initialized, Role: FOLLOWER
I20260811 22:24:05.496908 19485 consensus_queue.cc:260] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.497161 19485 ts_tablet_manager.cc:1428] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:24:05.498562 19989 tablet_service.cc:1467] Processing CreateTablet for tablet 96baa3ef8cd74efeb520782f900be57e (DEFAULT_TABLE table=test-table [id=bcd955bf37d845eaa7890f2ea047e72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:05.498533 19772 tablet_service.cc:1467] Processing CreateTablet for tablet 96baa3ef8cd74efeb520782f900be57e (DEFAULT_TABLE table=test-table [id=bcd955bf37d845eaa7890f2ea047e72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:05.499691 19772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96baa3ef8cd74efeb520782f900be57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:05.499698 19989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96baa3ef8cd74efeb520782f900be57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:05.506765 20116 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: Bootstrap starting.
I20260811 22:24:05.507405 20117 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: Bootstrap starting.
I20260811 22:24:05.510015 20117 tablet_bootstrap.cc:654] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:05.510015 20116 tablet_bootstrap.cc:654] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:05.511094 20117 log.cc:824] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:05.511094 20116 log.cc:824] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:05.513605 20117 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: No bootstrap required, opened a new log
I20260811 22:24:05.513605 20116 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: No bootstrap required, opened a new log
I20260811 22:24:05.513845 20117 ts_tablet_manager.cc:1397] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 22:24:05.513904 20116 ts_tablet_manager.cc:1397] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
W20260811 22:24:05.516307 19662 tablet.cc:2369] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:05.518286 20116 raft_consensus.cc:348] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.518285 20117 raft_consensus.cc:348] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.518676 20116 raft_consensus.cc:374] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:05.518676 20117 raft_consensus.cc:374] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:05.518739 20117 raft_consensus.cc:729] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f9ae43d7974caca32cf1b2608cfbb5, State: Initialized, Role: FOLLOWER
I20260811 22:24:05.518739 20116 raft_consensus.cc:729] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5c31bbd7a7e4709ab44fb091d28cf4a, State: Initialized, Role: FOLLOWER
I20260811 22:24:05.519347 20117 consensus_queue.cc:260] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.519346 20116 consensus_queue.cc:260] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.520205 20092 heartbeater.cc:503] Master 127.26.125.62:36329 was elected leader, sending a full tablet report...
I20260811 22:24:05.520465 19877 heartbeater.cc:503] Master 127.26.125.62:36329 was elected leader, sending a full tablet report...
I20260811 22:24:05.520704 20116 ts_tablet_manager.cc:1428] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260811 22:24:05.520704 20117 ts_tablet_manager.cc:1428] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:24:05.535903 20124 raft_consensus.cc:482] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:05.536159 20124 raft_consensus.cc:504] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.537433 20124 leader_election.cc:288] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653), f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629)
I20260811 22:24:05.541455 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.537682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:34653 (local address 127.26.125.3:59747)
0811 22:24:05.537917 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.537933 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:05.538036 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:05.538325 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:05.538327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:05.538345 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:05.538457 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.538464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.539337 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.539340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.540219 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.540230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.541015 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.541019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.541129 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.541158 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:24:05.541232 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 22:24:05.541310 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 22:24:05.541538 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.537825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:59747 (local address 127.26.125.2:34653)
0811 22:24:05.537965 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.537969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.537989 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:24:05.538170 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.538173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.538231 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.538311 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:05.538568 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.539231 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.540357 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.540890 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.541015 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.541072 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.541266 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.541402 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:24:05.542008 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.537896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:46161 (local address 127.26.125.1:46629)
0811 22:24:05.538012 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.538017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.538304 (+ 287us) server_negotiation.cc:407] Connection header received
0811 22:24:05.538489 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.538492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.538566 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.538652 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:05.538945 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.539540 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.540654 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.541228 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.541330 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.541489 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.541808 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.541890 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:24:05.542332 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.537753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46629 (local address 127.26.125.3:46161)
0811 22:24:05.538204 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.538218 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:05.538324 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:05.538681 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:05.538684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:05.538703 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:05.538822 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.538828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.539656 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.539659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.540515 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:05.540531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.541572 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.541576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:05.541668 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.541693 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:24:05.542149 (+ 456us) client_negotiation.cc:240] Negotiation successful
0811 22:24:05.542223 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":167,"threads_started":1}
I20260811 22:24:05.542280 19803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96baa3ef8cd74efeb520782f900be57e" candidate_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" is_pre_election: true
I20260811 22:24:05.542629 19803 raft_consensus.cc:2454] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5c31bbd7a7e4709ab44fb091d28cf4a in term 0.
I20260811 22:24:05.542914 19581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96baa3ef8cd74efeb520782f900be57e" candidate_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" is_pre_election: true
I20260811 22:24:05.543156 19905 leader_election.cc:302] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44f9ae43d7974caca32cf1b2608cfbb5, b5c31bbd7a7e4709ab44fb091d28cf4a; no voters:
I20260811 22:24:05.543226 19581 raft_consensus.cc:2454] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5c31bbd7a7e4709ab44fb091d28cf4a in term 0.
I20260811 22:24:05.543447 20124 raft_consensus.cc:2781] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:24:05.543540 20124 raft_consensus.cc:482] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:24:05.543594 20124 raft_consensus.cc:3037] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:05.544732 20124 raft_consensus.cc:504] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.545050 20124 leader_election.cc:288] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [CANDIDATE]: Term 1 election: Requested vote from peers 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653), f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629)
I20260811 22:24:05.545393 19803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96baa3ef8cd74efeb520782f900be57e" candidate_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f9ae43d7974caca32cf1b2608cfbb5"
I20260811 22:24:05.545408 19581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96baa3ef8cd74efeb520782f900be57e" candidate_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c1f08b3435460ea2ea7b91ad846a78"
I20260811 22:24:05.545506 19803 raft_consensus.cc:3037] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:05.545526 19581 raft_consensus.cc:3037] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:05.546530 19581 raft_consensus.cc:2454] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5c31bbd7a7e4709ab44fb091d28cf4a in term 1.
I20260811 22:24:05.546643 19803 raft_consensus.cc:2454] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5c31bbd7a7e4709ab44fb091d28cf4a in term 1.
I20260811 22:24:05.546828 19906 leader_election.cc:302] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5c31bbd7a7e4709ab44fb091d28cf4a, f2c1f08b3435460ea2ea7b91ad846a78; no voters:
I20260811 22:24:05.547034 20124 raft_consensus.cc:2781] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:05.547267 20124 raft_consensus.cc:686] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 LEADER]: Becoming Leader. State: Replica: b5c31bbd7a7e4709ab44fb091d28cf4a, State: Running, Role: LEADER
I20260811 22:24:05.547628 20124 consensus_queue.cc:237] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:05.551077 19347 catalog_manager.cc:5495] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a reported cstate change: term changed from 0 to 1, leader changed from <none> to b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3). New cstate: current_term: 1 leader_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } health_report { overall_health: UNKNOWN } } }
I20260811 22:24:05.562580 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.559316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.26.125.3:34611)
0811 22:24:05.559439 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.559443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.559458 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:24:05.559600 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.559602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.559660 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.559760 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:05.559964 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.560456 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.561243 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.561455 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.561536 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.561790 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:05.562259 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:05.562352 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.562381 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.562433 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:24:05.569022 19581 raft_consensus.cc:1261] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Refusing update from remote peer b5c31bbd7a7e4709ab44fb091d28cf4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:05.569026 19803 raft_consensus.cc:1261] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 1 FOLLOWER]: Refusing update from remote peer b5c31bbd7a7e4709ab44fb091d28cf4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:05.569748 20130 consensus_queue.cc:1035] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:05.569938 20124 consensus_queue.cc:1035] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:05.577003 20138 mvcc.cc:205] Tried to move back new op lower bound from 7317450938643144704 to 7317450938566201344. Current Snapshot: MvccSnapshot[applied={T|T < 7317450938643144704}]
I20260811 22:24:05.579432 20136 mvcc.cc:205] Tried to move back new op lower bound from 7317450938643144704 to 7317450938566201344. Current Snapshot: MvccSnapshot[applied={T|T < 7317450938643144704}]
I20260811 22:24:05.585309 19347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:24:05.585528 19347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:24:05.588227 19347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Sending DeleteTablet for 3 replicas of tablet 96baa3ef8cd74efeb520782f900be57e
I20260811 22:24:05.589112 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 19416
I20260811 22:24:05.589200 19545 tablet_service.cc:1514] Processing DeleteTablet for tablet 96baa3ef8cd74efeb520782f900be57e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:05 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35886
I20260811 22:24:05.589140 19989 tablet_service.cc:1514] Processing DeleteTablet for tablet 96baa3ef8cd74efeb520782f900be57e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:05 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:35942
I20260811 22:24:05.589262 19772 tablet_service.cc:1514] Processing DeleteTablet for tablet 96baa3ef8cd74efeb520782f900be57e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:05 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:32928
I20260811 22:24:05.596251 20149 tablet_replica.cc:330] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: stopping tablet replica
I20260811 22:24:05.596700 20149 raft_consensus.cc:2229] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:24:05.597085 20149 raft_consensus.cc:2258] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:05.598091 20148 tablet_replica.cc:330] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: stopping tablet replica
I20260811 22:24:05.598419 20148 raft_consensus.cc:2229] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:24:05.598819 20148 raft_consensus.cc:2258] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:05.606588 20148 ts_tablet_manager.cc:1905] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:05.610383 20148 ts_tablet_manager.cc:1918] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:05.610518 20148 log.cc:1196] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/96baa3ef8cd74efeb520782f900be57e
I20260811 22:24:05.610895 20148 ts_tablet_manager.cc:1939] T 96baa3ef8cd74efeb520782f900be57e P b5c31bbd7a7e4709ab44fb091d28cf4a: Deleting consensus metadata
I20260811 22:24:05.615079 19327 catalog_manager.cc:4841] TS b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611): tablet 96baa3ef8cd74efeb520782f900be57e (table test-table [id=bcd955bf37d845eaa7890f2ea047e72f]) successfully deleted
I20260811 22:24:05.617712 20149 ts_tablet_manager.cc:1905] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:05.620761 20149 ts_tablet_manager.cc:1918] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:05.620904 20149 log.cc:1196] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/96baa3ef8cd74efeb520782f900be57e
I20260811 22:24:05.621261 20149 ts_tablet_manager.cc:1939] T 96baa3ef8cd74efeb520782f900be57e P 44f9ae43d7974caca32cf1b2608cfbb5: Deleting consensus metadata
I20260811 22:24:05.622831 19326 catalog_manager.cc:4841] TS 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653): tablet 96baa3ef8cd74efeb520782f900be57e (table test-table [id=bcd955bf37d845eaa7890f2ea047e72f]) successfully deleted
W20260811 22:24:05.624444 19327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:46629 (error 108)
I20260811 22:24:05.624711 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 19666
I20260811 22:24:05.634296 19672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.633862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46629 (local address 127.0.0.1:35888)
0811 22:24:05.634107 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:05.634192 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260811 22:24:05.634502 19327 catalog_manager.cc:4584] TS f2c1f08b3435460ea2ea7b91ad846a78 (127.26.125.1:46629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96baa3ef8cd74efeb520782f900be57e: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46629: connect: Connection refused (error 111)
I20260811 22:24:05.644223 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 19879
I20260811 22:24:05.674957 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 19299
I20260811 22:24:05.689668 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36329
--webserver_interface=127.26.125.62
--webserver_port=41575
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:05.807387 20151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:05.807691 20151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:05.807732 20151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:05.811832 20151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:05.811908 20151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:05.811926 20151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:05.811941 20151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:05.811955 20151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:05.817915 20151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=41575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:05.819072 20151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:05.820145 20151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:05.829044 20162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:05.829030 20164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:05.829035 20161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:05.829201 20151 server_base.cc:1034] running on GCE node
I20260811 22:24:05.829901 20151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:05.831224 20151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:05.832391 20151 hybrid_clock.cc:648] HybridClock initialized: now 1786487045832367 us; error 73 us; skew 500 ppm
Aug 11 22:24:05 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487045, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:05.835274 20151 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:24:05.836277 20151 webserver.cc:460] Webserver started at http://127.26.125.62:41575/ using document root <none> and password file <none>
I20260811 22:24:05.836874 20151 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:05.836941 20151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:05.840989 20151 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:24:05.843262 20177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:05.844610 20151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:05.844725 20151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "d4d59c48f5d24a46b2aa332d90de4eed"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "4e90f89d3d4ee61d5f9a0a59fccc09d7"
server_key_iv: "f37e47d0ab01f700659d4dab21085641"
server_key_version: "encryptionkey@0"
I20260811 22:24:05.845304 20151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:05.867648 20151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:05.870844 20151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:05.871037 20151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:05.878324 20151 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:36329
I20260811 22:24:05.878641 20270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:36329 every 8 connection(s)
I20260811 22:24:05.879882 20151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:05.885931 20271 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:24:05.887492 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 20151
I20260811 22:24:05.887938 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:46629
--local_ip_for_outbound_sockets=127.26.125.1
--tserver_master_addrs=127.26.125.62:36329
--webserver_port=40905
--webserver_interface=127.26.125.1
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:05.890784 20271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Bootstrap starting.
I20260811 22:24:05.908699 20271 log.cc:824] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:05.920492 20271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:24:05.920869 20271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed: Bootstrap complete.
I20260811 22:24:05.924820 20271 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:05.925209 20271 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d59c48f5d24a46b2aa332d90de4eed, State: Initialized, Role: FOLLOWER
I20260811 22:24:05.925722 20271 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:05.925850 20271 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:05.925917 20271 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:05.926020 20271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:24:05.927990 20271 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:05.928306 20271 leader_election.cc:302] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4d59c48f5d24a46b2aa332d90de4eed; no voters:
I20260811 22:24:05.928643 20271 leader_election.cc:288] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:24:05.928747 20280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:24:05.928967 20280 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [term 3 LEADER]: Becoming Leader. State: Replica: d4d59c48f5d24a46b2aa332d90de4eed, State: Running, Role: LEADER
I20260811 22:24:05.929271 20280 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } }
I20260811 22:24:05.929945 20271 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:05.930300 20280 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4d59c48f5d24a46b2aa332d90de4eed. Latest consensus state: current_term: 3 leader_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } } }
I20260811 22:24:05.930589 20280 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:05.931286 20281 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d59c48f5d24a46b2aa332d90de4eed" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36329 } } }
I20260811 22:24:05.931464 20281 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d59c48f5d24a46b2aa332d90de4eed [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:05.931856 20285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:05.937476 20285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:05.938126 20285 catalog_manager.cc:1259] Loaded cluster ID: 6a479ffbbd6246bd94658ddd0c8e5c8e
I20260811 22:24:05.938187 20285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 22:24:05.939337 20285 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7effa5c1f9ad google::LogMessage::Fail() at ??:0
@ 0x7effa5c2288a google::LogMessage::SendToLog() at ??:0
@ 0x7effa5c1f4de google::LogMessage::Flush() at ??:0
@ 0x7effa5c20a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7effb055efdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7effb055fd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7effb055c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7effb05a0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7effa7e713d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7effa7e71c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7effa7e733f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7effa7e62512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7effa94a66db start_thread at ??:0
@ 0x7effa471d61f clone at ??:0
W20260811 22:24:06.034291 20276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:06.034595 20276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:06.034643 20276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:06.038822 20276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:06.038939 20276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:06.039026 20276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:24:06.044603 20276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:46629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=40905
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:06.045820 20276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:06.046957 20276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:06.056988 20338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:06.057126 20276 server_base.cc:1034] running on GCE node
W20260811 22:24:06.057338 20341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:06.056991 20339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:06.057870 20276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:06.058562 20276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:06.059731 20276 hybrid_clock.cc:648] HybridClock initialized: now 1786487046059687 us; error 62 us; skew 500 ppm
Aug 11 22:24:06 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487046, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:06.063153 20276 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:24:06.064122 20276 webserver.cc:460] Webserver started at http://127.26.125.1:40905/ using document root <none> and password file <none>
I20260811 22:24:06.064715 20276 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:06.064790 20276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:06.069131 20276 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:24:06.071631 20352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:06.073036 20276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:06.073165 20276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "f2c1f08b3435460ea2ea7b91ad846a78"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "d9820c319a832f1f4d7988cb702c990e"
server_key_iv: "50a06d4ebd22295bbe49587effdf7f80"
server_key_version: "encryptionkey@0"
I20260811 22:24:06.073722 20276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:06.087550 20276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:06.091389 20276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:06.091609 20276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:06.092314 20276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:06.093542 20366 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:24:06.099393 20276 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:24:06.099473 20276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 22:24:06.099545 20276 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:24:06.101370 20276 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:24:06.101421 20276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 22:24:06.101783 20366 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Bootstrap starting.
I20260811 22:24:06.111783 20276 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:46629
I20260811 22:24:06.112066 20544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:46629 every 8 connection(s)
I20260811 22:24:06.112859 20276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:24:06.117633 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 20276
I20260811 22:24:06.118060 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:34653
--local_ip_for_outbound_sockets=127.26.125.2
--tserver_master_addrs=127.26.125.62:36329
--webserver_port=39327
--webserver_interface=127.26.125.2
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:06.124204 20366 log.cc:824] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:06.134635 20366 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:24:06.135152 20366 tablet_bootstrap.cc:492] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Bootstrap complete.
I20260811 22:24:06.136433 20366 ts_tablet_manager.cc:1397] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Time spent bootstrapping tablet: real 0.035s user 0.029s sys 0.004s
I20260811 22:24:06.139899 20366 raft_consensus.cc:348] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:06.140484 20366 raft_consensus.cc:729] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c1f08b3435460ea2ea7b91ad846a78, State: Initialized, Role: FOLLOWER
I20260811 22:24:06.141062 20366 consensus_queue.cc:260] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:06.142212 20366 ts_tablet_manager.cc:1428] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 22:24:06.157022 20549 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:06.114881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:59101)
0811 22:24:06.115401 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.115458 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:06.116299 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:06.156164 (+ 39865us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":129,"threads_started":1}
W20260811 22:24:06.157590 20545 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:36329 (0 consecutive failures): Network error: Failed to ping master at 127.26.125.62:36329: Client connection negotiation failed: client connection to 127.26.125.62:36329: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:24:06.158368 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:06.158114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:48843)
0811 22:24:06.158238 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.158306 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 22:24:06.159088 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:06.158835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:37513)
0811 22:24:06.158983 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.159032 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 22:24:06.159260 20545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:24:06.242008 20550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:06.242440 20550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:06.242504 20550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:06.246719 20550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:06.246793 20550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:06.246884 20550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:24:06.252499 20550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:34653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=39327
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:06.253980 20550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:06.255151 20550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:06.265481 20563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:06.265451 20562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:06.265525 20567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:06.265738 20550 server_base.cc:1034] running on GCE node
I20260811 22:24:06.266150 20550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:06.266901 20550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:06.268082 20550 hybrid_clock.cc:648] HybridClock initialized: now 1786487046268047 us; error 58 us; skew 500 ppm
Aug 11 22:24:06 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487046, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:06.271636 20550 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:24:06.272656 20550 webserver.cc:460] Webserver started at http://127.26.125.2:39327/ using document root <none> and password file <none>
I20260811 22:24:06.273265 20550 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:06.273345 20550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:06.277890 20550 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:06.280366 20584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:06.281787 20550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:24:06.281913 20550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "44f9ae43d7974caca32cf1b2608cfbb5"
format_stamp: "Formatted at 2026-08-11 22:24:03 on dist-test-slave-5bn2"
server_key: "cce361a0de9f2f647c58489b2f140783"
server_key_iv: "54b42fd8f8e6f13213d901e63cf5c160"
server_key_version: "encryptionkey@0"
I20260811 22:24:06.282523 20550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:06.309577 20550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:06.312858 20550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:06.313045 20550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:06.313735 20550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:06.314630 20550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:06.314679 20550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:06.314726 20550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:06.314756 20550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:06.323966 20550 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:34653
I20260811 22:24:06.324000 20777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:34653 every 8 connection(s)
I20260811 22:24:06.325031 20550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:24:06.327988 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 20550
I20260811 22:24:06.328480 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:34611
--local_ip_for_outbound_sockets=127.26.125.3
--tserver_master_addrs=127.26.125.62:36329
--webserver_port=39945
--webserver_interface=127.26.125.3
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:06.329495 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:06.327502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:41701)
0811 22:24:06.328117 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.328629 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":157,"threads_started":1}
W20260811 22:24:06.330018 20779 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:36329 (0 consecutive failures): Network error: Failed to ping master at 127.26.125.62:36329: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
I20260811 22:24:06.330749 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:06.330538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:38307)
0811 22:24:06.330641 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.330691 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 22:24:06.331387 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:06.331181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:49465)
0811 22:24:06.331286 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.331328 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:24:06.331589 20779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:24:06.447994 20785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:06.448307 20785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:06.448367 20785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:06.452486 20785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:06.452566 20785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:06.452648 20785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:24:06.458335 20785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:34611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=39945
--tserver_master_addrs=127.26.125.62:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:06.459525 20785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:06.460638 20785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:06.470908 20801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:06.470872 20797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:06.471031 20785 server_base.cc:1034] running on GCE node
W20260811 22:24:06.472031 20794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:06.472529 20785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:06.473220 20785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:06.474445 20785 hybrid_clock.cc:648] HybridClock initialized: now 1786487046474410 us; error 45 us; skew 500 ppm
Aug 11 22:24:06 dist-test-slave-5bn2 krb5kdc[18437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487046, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:06.477545 20785 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:24:06.478569 20785 webserver.cc:460] Webserver started at http://127.26.125.3:39945/ using document root <none> and password file <none>
I20260811 22:24:06.479211 20785 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:06.479282 20785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:06.483461 20785 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:24:06.485978 20813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:06.487526 20785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:06.487668 20785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a"
format_stamp: "Formatted at 2026-08-11 22:24:04 on dist-test-slave-5bn2"
server_key: "adbe2ec992a9fd6d0ea65f7082aa10a6"
server_key_iv: "87707b1e085407826417517bf3e5d5b9"
server_key_version: "encryptionkey@0"
I20260811 22:24:06.488272 20785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:06.514928 20785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:06.518476 20785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:06.518683 20785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:06.519399 20785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:06.520303 20785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:06.520354 20785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:06.520409 20785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:06.520438 20785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:06.529296 20785 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:34611
I20260811 22:24:06.529332 21025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:34611 every 8 connection(s)
I20260811 22:24:06.530398 20785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:24:06.534835 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:06.532994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:35979)
0811 22:24:06.533532 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.534017 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":126,"threads_started":1}
W20260811 22:24:06.535398 21029 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:36329 (0 consecutive failures): Network error: Failed to ping master at 127.26.125.62:36329: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
I20260811 22:24:06.536067 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:06.535838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:44881)
0811 22:24:06.535947 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.536001 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 22:24:06.536702 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:06.536535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:49655)
0811 22:24:06.536620 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:06.536659 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 22:24:06.536867 21029 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:24:06.537182 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 20785
I20260811 22:24:07.160269 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.159823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:48191)
0811 22:24:07.160082 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:07.160143 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260811 22:24:07.332647 21037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.332212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:49877)
0811 22:24:07.332465 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:07.332531 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260811 22:24:07.537981 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.537526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:53205)
0811 22:24:07.537793 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:07.537858 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
I20260811 22:24:07.761937 21041 raft_consensus.cc:482] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:07.762161 21041 raft_consensus.cc:504] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:07.763501 21041 leader_election.cc:288] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611), 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
Aug 11 22:24:07 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487046, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.3@KRBTEST.COM
Aug 11 22:24:07 dist-test-slave-5bn2 krb5kdc[18437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487046, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.2@KRBTEST.COM
I20260811 22:24:07.777757 21050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.764152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:49299 (local address 127.26.125.3:34611)
0811 22:24:07.764500 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:07.764527 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:07.764605 (+ 78us) server_negotiation.cc:407] Connection header received
0811 22:24:07.765321 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:07.765327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:07.765677 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:07.765991 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:07.767438 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:07.768506 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:07.769683 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:07.770191 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:07.770348 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:07.773113 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:07.773119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:07.773133 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:07.773166 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:07.775004 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:07.775609 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:07.775616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:07.775622 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:07.775751 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:07.776086 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:07.776089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:07.776091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:07.776533 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:07.776689 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:07.777281 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 22:24:07.777563 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":63,"threads_started":1}
I20260811 22:24:07.777834 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.763990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:39849 (local address 127.26.125.2:34653)
0811 22:24:07.764362 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:07.764393 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:07.764527 (+ 134us) server_negotiation.cc:407] Connection header received
0811 22:24:07.765267 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:07.765274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:07.765676 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:07.765992 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:07.767437 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:07.768526 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:07.769701 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:07.770223 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:07.770380 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:07.773366 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:07.773371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:07.773386 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:07.773417 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:07.775056 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:07.775720 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:07.775725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:07.775730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:07.775872 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:07.776159 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:07.776162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:07.776163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:07.776600 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:07.776743 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:07.777284 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 22:24:07.777641 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
I20260811 22:24:07.778133 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.763834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:34611 (local address 127.26.125.1:49299)
0811 22:24:07.764443 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:07.764469 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:07.764604 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:07.766108 (+ 1504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:07.766111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:07.766688 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:07.767236 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:07.767251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:07.768659 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:07.768662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:07.769552 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:07.769572 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:07.770313 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:07.770316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:07.770517 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:07.771235 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:07.771253 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:07.772859 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:07.775125 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:07.775132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:07.775148 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:07.775492 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:07.775842 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:07.775845 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:07.775847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:07.775985 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:07.776680 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:07.776687 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:07.776967 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 22:24:07.777143 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 22:24:07.777610 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":702,"spinlock_wait_cycles":19712,"thread_start_us":69,"threads_started":1}
I20260811 22:24:07.777778 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.763834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:34653 (local address 127.26.125.1:39849)
0811 22:24:07.764364 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:07.764380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:07.764516 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:07.766102 (+ 1586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:07.766109 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:07.766688 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:07.767233 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:07.767250 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:07.768654 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:07.768658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:07.769555 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:07.769567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:07.770346 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:07.770350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:07.770564 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:07.771256 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:07.771269 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:07.773169 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:07.775176 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:07.775180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:07.775182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:07.775541 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:07.775983 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:07.775986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:07.775988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:07.776073 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:07.776710 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:07.776713 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:07.776966 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 22:24:07.777143 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 22:24:07.777513 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":229,"spinlock_wait_cycles":14464,"thread_start_us":102,"threads_started":1}
W20260811 22:24:07.781086 20357 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
W20260811 22:24:07.781319 20358 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
I20260811 22:24:07.781383 20358 leader_election.cc:302] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2c1f08b3435460ea2ea7b91ad846a78; no voters: 44f9ae43d7974caca32cf1b2608cfbb5, b5c31bbd7a7e4709ab44fb091d28cf4a
I20260811 22:24:07.781662 21041 raft_consensus.cc:2726] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:24:08.161469 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:08.161082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:49971)
0811 22:24:08.161246 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:08.161316 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 22:24:08.333873 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:08.333422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:58601)
0811 22:24:08.333703 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:08.333768 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":93,"threads_started":1}
I20260811 22:24:08.539498 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:08.538996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:35669)
0811 22:24:08.539324 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:08.539393 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":78,"threads_started":1}
I20260811 22:24:09.162818 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:09.162330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:48509)
0811 22:24:09.162642 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:09.162711 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":102,"threads_started":1}
I20260811 22:24:09.335213 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:09.334698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:33589)
0811 22:24:09.335036 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:09.335094 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":96,"threads_started":1}
I20260811 22:24:09.540905 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:09.540362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:54225)
0811 22:24:09.540704 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:09.540774 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
I20260811 22:24:10.014122 21058 raft_consensus.cc:482] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:10.014267 21058 raft_consensus.cc:504] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:10.014652 21058 leader_election.cc:288] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611), 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
W20260811 22:24:10.015484 20358 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
W20260811 22:24:10.015596 20357 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
I20260811 22:24:10.015686 20357 leader_election.cc:302] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2c1f08b3435460ea2ea7b91ad846a78; no voters: 44f9ae43d7974caca32cf1b2608cfbb5, b5c31bbd7a7e4709ab44fb091d28cf4a
I20260811 22:24:10.015826 21058 raft_consensus.cc:2726] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:24:10.164263 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:10.163749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:36597)
0811 22:24:10.164053 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:10.164133 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":100,"threads_started":1}
I20260811 22:24:10.336416 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:10.336002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:38907)
0811 22:24:10.336264 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:10.336318 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":89,"threads_started":1}
I20260811 22:24:10.542234 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:10.541731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:53247)
0811 22:24:10.542061 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:10.542132 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":81,"threads_started":1}
I20260811 22:24:11.165930 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:11.165330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:37037)
0811 22:24:11.165722 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:11.165806 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":95,"threads_started":1}
I20260811 22:24:11.337673 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:11.337226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:49495)
0811 22:24:11.337502 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:11.337567 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260811 22:24:11.543521 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:11.543034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:48845)
0811 22:24:11.543303 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:11.543364 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260811 22:24:12.167260 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:12.166764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:51155)
0811 22:24:12.167077 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:12.167159 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":118,"threads_started":1}
I20260811 22:24:12.288679 21076 raft_consensus.cc:482] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:12.288875 21076 raft_consensus.cc:504] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:12.289942 21076 leader_election.cc:288] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611), 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
W20260811 22:24:12.290932 20357 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
W20260811 22:24:12.291146 20358 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
I20260811 22:24:12.291245 20358 leader_election.cc:302] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2c1f08b3435460ea2ea7b91ad846a78; no voters: 44f9ae43d7974caca32cf1b2608cfbb5, b5c31bbd7a7e4709ab44fb091d28cf4a
I20260811 22:24:12.291416 21076 raft_consensus.cc:2726] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:24:12.338990 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:12.338513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:37497)
0811 22:24:12.338779 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:12.338872 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260811 22:24:12.544768 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:12.544290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:60695)
0811 22:24:12.544590 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:12.544661 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":96,"threads_started":1}
I20260811 22:24:13.168517 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:13.168035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:39919)
0811 22:24:13.168335 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:13.168402 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":88,"threads_started":1}
I20260811 22:24:13.340507 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:13.339906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:52361)
0811 22:24:13.340315 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:13.340385 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":116,"threads_started":1}
I20260811 22:24:13.546061 21085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:13.545575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:54749)
0811 22:24:13.545897 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:13.545958 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":77,"threads_started":1}
I20260811 22:24:14.169857 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:14.169393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:49529)
0811 22:24:14.169683 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:14.169750 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
I20260811 22:24:14.341840 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:14.341359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:39705)
0811 22:24:14.341673 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:14.341738 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":78,"threads_started":1}
I20260811 22:24:14.547412 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:14.546987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:53913)
0811 22:24:14.547251 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:14.547314 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260811 22:24:15.171187 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:15.170714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:60161)
0811 22:24:15.171011 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:15.171080 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":85,"threads_started":1}
I20260811 22:24:15.343240 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:15.342736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:57837)
0811 22:24:15.343063 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:15.343133 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":114,"threads_started":1}
I20260811 22:24:15.548758 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:15.548285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:35167)
0811 22:24:15.548586 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:15.548653 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
I20260811 22:24:15.997186 21097 raft_consensus.cc:482] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:15.997339 21097 raft_consensus.cc:504] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:15.997800 21097 leader_election.cc:288] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611), 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
W20260811 22:24:15.998582 20357 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
W20260811 22:24:15.998678 20358 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
I20260811 22:24:15.998790 20358 leader_election.cc:302] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2c1f08b3435460ea2ea7b91ad846a78; no voters: 44f9ae43d7974caca32cf1b2608cfbb5, b5c31bbd7a7e4709ab44fb091d28cf4a
I20260811 22:24:15.998977 21097 raft_consensus.cc:2726] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:24:16.172664 21098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:16.172166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:40173)
0811 22:24:16.172465 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:16.172533 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":86,"threads_started":1}
I20260811 22:24:16.344480 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:16.344034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:43847)
0811 22:24:16.344303 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:16.344365 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":76,"threads_started":1}
I20260811 22:24:16.550287 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:16.549755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:55647)
0811 22:24:16.550099 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:16.550180 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":97,"threads_started":1}
I20260811 22:24:17.174024 21102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:17.173536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:39955)
0811 22:24:17.173843 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:17.173916 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":81,"threads_started":1}
I20260811 22:24:17.345695 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:17.345254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:52325)
0811 22:24:17.345519 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:17.345586 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":79,"threads_started":1}
I20260811 22:24:17.551608 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:17.551110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:55143)
0811 22:24:17.551434 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:17.551500 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":95,"threads_started":1}
I20260811 22:24:17.570309 21105 raft_consensus.cc:482] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:17.570477 21105 raft_consensus.cc:504] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5c31bbd7a7e4709ab44fb091d28cf4a" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34611 } } peers { permanent_uuid: "44f9ae43d7974caca32cf1b2608cfbb5" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 34653 } } peers { permanent_uuid: "f2c1f08b3435460ea2ea7b91ad846a78" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46629 } }
I20260811 22:24:17.571010 21105 leader_election.cc:288] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611), 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653)
W20260811 22:24:17.571841 20358 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5c31bbd7a7e4709ab44fb091d28cf4a (127.26.125.3:34611): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
W20260811 22:24:17.571955 20357 leader_election.cc:341] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f9ae43d7974caca32cf1b2608cfbb5 (127.26.125.2:34653): Not found: Tablet not found: 96baa3ef8cd74efeb520782f900be57e
I20260811 22:24:17.572026 20357 leader_election.cc:302] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2c1f08b3435460ea2ea7b91ad846a78; no voters: 44f9ae43d7974caca32cf1b2608cfbb5, b5c31bbd7a7e4709ab44fb091d28cf4a
I20260811 22:24:17.572165 21105 raft_consensus.cc:2726] T 96baa3ef8cd74efeb520782f900be57e P f2c1f08b3435460ea2ea7b91ad846a78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:24:18.175319 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:18.174824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:44261)
0811 22:24:18.175144 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:18.175214 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":79,"threads_started":1}
I20260811 22:24:18.347066 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:18.346524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:45159)
0811 22:24:18.346839 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:18.346958 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":89,"threads_started":1}
I20260811 22:24:18.552973 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:18.552511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:54619)
0811 22:24:18.552798 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:18.552864 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
I20260811 22:24:19.176728 21110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:19.176234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:56879)
0811 22:24:19.176555 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:19.176624 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":110,"threads_started":1}
I20260811 22:24:19.348305 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:19.347868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:39393)
0811 22:24:19.348142 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:19.348204 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
I20260811 22:24:19.554442 21113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:19.553897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:44733)
0811 22:24:19.554216 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:19.554287 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":107,"threads_started":1}
I20260811 22:24:20.177918 21114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:20.177505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:59673)
0811 22:24:20.177761 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:20.177816 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":80,"threads_started":1}
I20260811 22:24:20.349659 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:20.349176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:39665)
0811 22:24:20.349459 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:20.349529 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
I20260811 22:24:20.555819 21116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:20.555292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:52167)
0811 22:24:20.555640 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:20.555712 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":83,"threads_started":1}
I20260811 22:24:21.179327 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:21.178893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.1:58983)
0811 22:24:21.179161 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:21.179225 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260811 22:24:21.351217 21121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:21.350735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.2:55601)
0811 22:24:21.351045 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:21.351115 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":80,"threads_started":1}
I20260811 22:24:21.548604 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 20276
I20260811 22:24:21.570452 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:21.569991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:36329 (local address 127.26.125.3:33935)
0811 22:24:21.570283 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:21.570350 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":102,"threads_started":1}
I20260811 22:24:21.575152 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 20550
I20260811 22:24:21.586937 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 20785
2026-08-11T22:24:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19705 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:21 dist-test-slave-5bn2 krb5kdc[21126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:21 dist-test-slave-5bn2 krb5kdc[21126](info): set up 2 sockets
Aug 11 22:24:21 dist-test-slave-5bn2 krb5kdc[21126](info): commencing operation
krb5kdc: starting...
W20260811 22:24:22.799750 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:24:22 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:24:22.932222 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44939
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:23.048233 21150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:23.048565 21150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:23.048625 21150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:23.052862 21150 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:24:23.052938 21150 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:24:23.052966 21150 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:24:23.052983 21150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:23.053009 21150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:23.053042 21150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:23.053073 21150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:23.053099 21150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:23.058686 21150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:23.059888 21150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:23.060978 21150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:23.070564 21161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:23.070663 21158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:23.070689 21150 server_base.cc:1034] running on GCE node
W20260811 22:24:23.070590 21159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:23.071699 21150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:23.072899 21150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:23.074081 21150 hybrid_clock.cc:648] HybridClock initialized: now 1786487063074052 us; error 41 us; skew 500 ppm
Aug 11 22:24:23 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487063, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:23.077086 21150 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:24:23.078133 21150 webserver.cc:460] Webserver started at http://127.26.125.62:37207/ using document root <none> and password file <none>
I20260811 22:24:23.078744 21150 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:23.078820 21150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:23.079078 21150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:23.081357 21150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d8ed4d8f42e51c6fa808db4be85985c7"
server_key_iv: "bdbbc4f437dc70b6016184fdd8063d31"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.081956 21150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d8ed4d8f42e51c6fa808db4be85985c7"
server_key_iv: "bdbbc4f437dc70b6016184fdd8063d31"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.086277 21150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 22:24:23.089113 21175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.090646 21150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:24:23.090812 21150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d8ed4d8f42e51c6fa808db4be85985c7"
server_key_iv: "bdbbc4f437dc70b6016184fdd8063d31"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.090988 21150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:23.118728 21150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:23.122241 21150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:23.122448 21150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:23.129771 21150 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:44939
I20260811 22:24:23.130096 21285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:44939 every 8 connection(s)
I20260811 22:24:23.130918 21150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:23.133968 21286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:23.139962 21286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Bootstrap starting.
I20260811 22:24:23.140434 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 21150
I20260811 22:24:23.140553 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:24:23.140820 27124 external_mini_cluster.cc:1428] Setting key f2c767a568cf36458222f161c273afed
I20260811 22:24:23.142827 21286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:23.143839 21286 log.cc:824] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:23.145840 21286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: No bootstrap required, opened a new log
I20260811 22:24:23.149586 21286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:24:23.149843 21286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:23.149914 21286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c45d0b5bf7e4519b5c047b7601b9d5b, State: Initialized, Role: FOLLOWER
Aug 11 22:24:23 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:23.150424 21286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:24:23.150584 21286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:23.150657 21286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:23.150763 21286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:23.151907 21286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:24:23.152276 21286 leader_election.cc:302] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c45d0b5bf7e4519b5c047b7601b9d5b; no voters:
I20260811 22:24:23.152639 21286 leader_election.cc:288] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:23.152765 21295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:23.152999 21295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 1 LEADER]: Becoming Leader. State: Replica: 6c45d0b5bf7e4519b5c047b7601b9d5b, State: Running, Role: LEADER
I20260811 22:24:23.153366 21295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:24:23.154127 21286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:23.155155 21298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } } }
I20260811 22:24:23.155290 21298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:23.155890 21299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c45d0b5bf7e4519b5c047b7601b9d5b. Latest consensus state: current_term: 1 leader_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } } }
I20260811 22:24:23.156049 21299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:23.156378 21308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:23.156776 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.142238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53892 (local address 127.26.125.62:44939)
0811 22:24:23.142762 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:23.142785 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:23.142835 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:24:23.143750 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:23.143775 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:23.144255 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:23.144598 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:23.145419 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.146487 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.147414 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.147913 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.148090 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.150672 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:23.150680 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:23.150701 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:23.150741 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:23.152697 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.153258 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.153265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.153272 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.153426 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.153755 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.153759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.153761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.154331 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:23.154555 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:23.155307 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 22:24:23.155654 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":134,"threads_started":1}
I20260811 22:24:23.159869 21308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:23.165516 21308 catalog_manager.cc:1347] Generated new cluster ID: 67c27cc5383640f986db59de45b42ca3
I20260811 22:24:23.165606 21308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:23.188030 21308 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:23.189254 21308 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:23.200913 21308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Generated new TSK 0
I20260811 22:24:23.201891 21308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:24:23.237928 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44939
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:23.361424 21326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:23.361770 21326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:23.361838 21326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:23.366092 21326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:23.366168 21326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:23.366250 21326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:24:23.371901 21326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:23.373167 21326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:23.374413 21326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:24:23.384714 21326 server_base.cc:1034] running on GCE node
W20260811 22:24:23.384660 21339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:23.384670 21333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:23.384658 21334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:23.385439 21326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:23.386211 21326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:23.387425 21326 hybrid_clock.cc:648] HybridClock initialized: now 1786487063387401 us; error 35 us; skew 500 ppm
Aug 11 22:24:23 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487063, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:23.390542 21326 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:24:23.391544 21326 webserver.cc:460] Webserver started at http://127.26.125.1:33159/ using document root <none> and password file <none>
I20260811 22:24:23.392109 21326 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:23.392172 21326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:23.392347 21326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:23.394502 21326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "ba8a171dfbb14cc38eb02d5506ae3456"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "cf0c991e6fabf2921d7af78f72506b07"
server_key_iv: "eee5f2e81296a1a23ea935cf52083dbe"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.395102 21326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "ba8a171dfbb14cc38eb02d5506ae3456"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "cf0c991e6fabf2921d7af78f72506b07"
server_key_iv: "eee5f2e81296a1a23ea935cf52083dbe"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.399482 21326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:24:23.402253 21353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.403801 21326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:23.403967 21326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "ba8a171dfbb14cc38eb02d5506ae3456"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "cf0c991e6fabf2921d7af78f72506b07"
server_key_iv: "eee5f2e81296a1a23ea935cf52083dbe"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.404089 21326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:23.420081 21326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:23.423568 21326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:23.423768 21326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:23.424399 21326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:23.425340 21326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:23.425395 21326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.425451 21326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:23.425477 21326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.434602 21326 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:44389
I20260811 22:24:23.434643 21558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:44389 every 8 connection(s)
I20260811 22:24:23.435613 21326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:24:23.436642 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 21326
I20260811 22:24:23.436766 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:24:23.437053 27124 external_mini_cluster.cc:1428] Setting key e526b3344581d8b83750dda5587a412d
Aug 11 22:24:23 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487063, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:24:23.450811 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.437662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:55357 (local address 127.26.125.62:44939)
0811 22:24:23.437800 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:23.437805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:23.439486 (+ 1681us) server_negotiation.cc:407] Connection header received
0811 22:24:23.440700 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:23.440704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:23.440757 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:23.440861 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:23.442293 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.442836 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.444119 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.444364 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.444455 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.447036 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:23.447039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:23.447042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:23.447069 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:23.448473 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.449031 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.449034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.449036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.449124 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.449473 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.449476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.449479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.449723 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:23.449817 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:23.450469 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 22:24:23.450629 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:24:23.451709 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.438346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:55357)
0811 22:24:23.439295 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:23.439360 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:23.440405 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:23.441112 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:23.441120 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:23.441556 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:23.442125 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.442141 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.442988 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.442992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.443973 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.443996 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.444520 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.444525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.444702 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.445300 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:23.445319 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:23.446821 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:23.448612 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:23.448619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:23.448641 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:23.448928 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:23.449212 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:23.449216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:23.449218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:23.449381 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:23.449850 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:23.449857 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:23.450166 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 22:24:23.450406 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 22:24:23.450784 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":150,"threads_started":1}
I20260811 22:24:23.453209 21559 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44939
I20260811 22:24:23.453495 21559 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:23.454087 21559 heartbeater.cc:511] Master 127.26.125.62:44939 requested a full tablet report, sending...
I20260811 22:24:23.456257 21211 ts_manager.cc:194] Registered new tserver with Master: ba8a171dfbb14cc38eb02d5506ae3456 (127.26.125.1:44389)
I20260811 22:24:23.457942 21211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:55357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:24:23.469777 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44939
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:23.590586 21567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:23.590942 21567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:23.590989 21567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:23.595362 21567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:23.595458 21567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:23.595522 21567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:24:23.601217 21567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:23.602394 21567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:23.603505 21567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:23.614248 21574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:23.614387 21567 server_base.cc:1034] running on GCE node
W20260811 22:24:23.614249 21580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:23.614248 21575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:23.615052 21567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:23.615796 21567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:23.616966 21567 hybrid_clock.cc:648] HybridClock initialized: now 1786487063616940 us; error 46 us; skew 500 ppm
Aug 11 22:24:23 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487063, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:23.619881 21567 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:24:23.620802 21567 webserver.cc:460] Webserver started at http://127.26.125.2:34961/ using document root <none> and password file <none>
I20260811 22:24:23.621356 21567 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:23.621416 21567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:23.621604 21567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:23.623741 21567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "c8976bed2865d2be27c73254d2c4f8a8"
server_key_iv: "135b6a40f47d156d4398a789b39bdd27"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.624280 21567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "c8976bed2865d2be27c73254d2c4f8a8"
server_key_iv: "135b6a40f47d156d4398a789b39bdd27"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.628134 21567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:24:23.630746 21591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.632243 21567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:24:23.632362 21567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "c8976bed2865d2be27c73254d2c4f8a8"
server_key_iv: "135b6a40f47d156d4398a789b39bdd27"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.632493 21567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:23.645777 21567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:23.649230 21567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:23.649420 21567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:23.650077 21567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:23.651063 21567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:23.651116 21567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.651165 21567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:23.651193 21567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.660413 21567 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:40803
I20260811 22:24:23.660439 21818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:40803 every 8 connection(s)
I20260811 22:24:23.661412 21567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:24:23.668084 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 21567
I20260811 22:24:23.668210 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:24:23.668483 27124 external_mini_cluster.cc:1428] Setting key e2bd41c7024ff8940ded187ef8eed282
Aug 11 22:24:23 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487063, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:23.675563 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.663400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:56967 (local address 127.26.125.62:44939)
0811 22:24:23.663543 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:23.663547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:23.664638 (+ 1091us) server_negotiation.cc:407] Connection header received
0811 22:24:23.665542 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:23.665544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:23.665595 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:23.665667 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:23.666967 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.667509 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.668815 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.669110 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.669205 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.672030 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:23.672033 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:23.672035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:23.672062 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:23.673420 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.673940 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.673943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.673945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.674035 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.674366 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.674370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.674372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.674574 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:23.674663 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:23.675265 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 22:24:23.675395 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:24:23.676578 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.663828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:56967)
0811 22:24:23.664495 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:23.664544 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:23.665352 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:23.665837 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:23.665844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:23.666258 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:23.666792 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.666809 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.667646 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.667649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.668678 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.668690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.669267 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.669272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.669453 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.670026 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:23.670043 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:23.671831 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:23.673541 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:23.673548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:23.673568 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:23.673842 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:23.674121 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:23.674125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:23.674127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:23.674277 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:23.674677 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:23.674684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:23.675021 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 22:24:23.675206 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 22:24:23.675582 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":104,"threads_started":1}
I20260811 22:24:23.677999 21819 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44939
I20260811 22:24:23.678259 21819 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:24:23.678823 21819 heartbeater.cc:511] Master 127.26.125.62:44939 requested a full tablet report, sending...
I20260811 22:24:23.680121 21211 ts_manager.cc:194] Registered new tserver with Master: 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803)
I20260811 22:24:23.680776 21211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:56967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:24:23.699366 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44939
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:23.818099 21828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:23.818392 21828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:23.818434 21828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:23.822468 21828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:23.822538 21828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:23.822605 21828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:24:23.828073 21828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:23.829169 21828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:23.830229 21828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:23.840102 21838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:23.840176 21834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:23.840169 21828 server_base.cc:1034] running on GCE node
W20260811 22:24:23.840096 21835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:23.840703 21828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:23.841395 21828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:23.842559 21828 hybrid_clock.cc:648] HybridClock initialized: now 1786487063842540 us; error 32 us; skew 500 ppm
Aug 11 22:24:23 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487063, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:23.845238 21828 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:24:23.846132 21828 webserver.cc:460] Webserver started at http://127.26.125.3:40395/ using document root <none> and password file <none>
I20260811 22:24:23.846693 21828 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:23.846761 21828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:23.847028 21828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:23.849107 21828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "1977726fa0a94c7d85b35bc9d54ce61c"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d57245a3360de3eaafce5f80c42edb99"
server_key_iv: "6b980bd91dfddc0b9f3ee7214740d059"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.849695 21828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "1977726fa0a94c7d85b35bc9d54ce61c"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d57245a3360de3eaafce5f80c42edb99"
server_key_iv: "6b980bd91dfddc0b9f3ee7214740d059"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.853574 21828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:24:23.856274 21849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.857690 21828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:24:23.857816 21828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "1977726fa0a94c7d85b35bc9d54ce61c"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d57245a3360de3eaafce5f80c42edb99"
server_key_iv: "6b980bd91dfddc0b9f3ee7214740d059"
server_key_version: "encryptionkey@0"
I20260811 22:24:23.857932 21828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:23.880998 21828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:23.885043 21828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:23.885262 21828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:23.885939 21828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:23.886826 21828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:23.886909 21828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.886968 21828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:23.886991 21828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:23.895834 21828 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:45463
I20260811 22:24:23.895855 22047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:45463 every 8 connection(s)
I20260811 22:24:23.896920 21828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:24:23.897971 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 21828
I20260811 22:24:23.898118 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:24:23.898463 27124 external_mini_cluster.cc:1428] Setting key ff586f891c27c9c085e475aaee04f1b3
Aug 11 22:24:23 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487063, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:23.910816 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.899220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:44331 (local address 127.26.125.62:44939)
0811 22:24:23.899365 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:23.899370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:23.900551 (+ 1181us) server_negotiation.cc:407] Connection header received
0811 22:24:23.901509 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:23.901512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:23.901558 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:23.901624 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:23.902873 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.903422 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.904553 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.904762 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.904849 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.907393 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:23.907396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:23.907398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:23.907425 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:23.908723 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.909251 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.909254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.909255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.909346 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.909689 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.909692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.909693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.909898 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:23.909987 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:23.910561 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 22:24:23.910686 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:24:23.911772 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.899632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:44331)
0811 22:24:23.900387 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:23.900449 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:23.901313 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:23.901821 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:23.901830 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:23.902213 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:23.902729 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.902743 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.903532 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.903535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.904453 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.904463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.904883 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.904886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.905033 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.905587 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:23.905605 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:23.907212 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:23.908828 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:23.908834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:23.908848 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:23.909134 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:23.909431 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:23.909433 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:23.909434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:23.909590 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:23.909976 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:23.909981 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:23.910255 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 22:24:23.910429 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 22:24:23.910739 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":111,"threads_started":1}
I20260811 22:24:23.913020 22048 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44939
I20260811 22:24:23.913264 22048 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:23.913739 22048 heartbeater.cc:511] Master 127.26.125.62:44939 requested a full tablet report, sending...
I20260811 22:24:23.914785 21211 ts_manager.cc:194] Registered new tserver with Master: 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
I20260811 22:24:23.915438 21211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:44331
I20260811 22:24:23.924767 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:24:23.934041 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.926881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53904 (local address 127.26.125.62:44939)
0811 22:24:23.927049 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:23.927054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:23.927168 (+ 114us) server_negotiation.cc:407] Connection header received
0811 22:24:23.927312 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:23.927315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:23.927361 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:23.927422 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:23.928152 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.928686 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.929535 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.929752 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.929843 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.931251 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:23.931256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:23.931258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:23.931286 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:23.932381 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.932787 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.932791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.932792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.932886 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:23.933135 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:23.933138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:23.933140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:23.933382 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:23.933503 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:23.933751 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 22:24:23.933873 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:24:23.937121 21211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:24:23.940232 21211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:24:23.955933 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.950655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37496 (local address 127.26.125.3:45463)
0811 22:24:23.951178 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:23.951189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:23.951208 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:24:23.951273 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:23.951277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:23.951421 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:23.951562 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:23.951837 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.952930 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.954224 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.955041 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.955146 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.955365 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:23.955652 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:24:23.955768 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":210,"threads_started":1}
I20260811 22:24:23.955931 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.950210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:45463 (local address 127.0.0.1:37496)
0811 22:24:23.950872 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:23.950888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:23.950991 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:23.951557 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:23.951560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:23.951580 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:23.951717 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.951726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.953080 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.953084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.954098 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.954109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.955194 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.955199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.955325 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.955392 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 22:24:23.955584 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 22:24:23.955748 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":7,"thread_start_us":99,"threads_started":1}
I20260811 22:24:23.956103 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.949322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:40803 (local address 127.0.0.1:43850)
0811 22:24:23.949897 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:23.949959 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:23.950136 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:23.950997 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:23.951004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:23.951028 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:23.951184 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.951191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.952870 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.952874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.954496 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.954509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.955715 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.955719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.955819 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.955843 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:24:23.955890 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 22:24:23.955962 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":119,"thread_start_us":118,"threads_started":1}
I20260811 22:24:23.956498 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.950000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58190 (local address 127.26.125.1:44389)
0811 22:24:23.950423 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:23.950438 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:23.950738 (+ 300us) server_negotiation.cc:407] Connection header received
0811 22:24:23.950983 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:23.950989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:23.951197 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:23.951330 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:23.951781 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.952633 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.954681 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.955535 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.955638 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.955791 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:23.956166 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 22:24:23.956277 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":94,"threads_started":1}
I20260811 22:24:23.956866 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.949421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43850 (local address 127.26.125.2:40803)
0811 22:24:23.950423 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:23.950436 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:23.950459 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:24:23.950543 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:23.950549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:23.950802 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:23.951334 (+ 532us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:23.951494 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.952723 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.954640 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.955583 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.956323 (+ 740us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.956513 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:23.956629 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 22:24:23.956722 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":145,"threads_started":1}
I20260811 22:24:23.956887 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:23.949719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:44389 (local address 127.0.0.1:58190)
0811 22:24:23.950644 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:23.950657 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:23.950804 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:23.951527 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:23.951530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:23.951548 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:23.951654 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.951661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.952917 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.952920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.954532 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:23.954541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:23.955872 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:23.955876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:23.955984 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:23.956011 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:24:23.956656 (+ 645us) client_negotiation.cc:240] Negotiation successful
0811 22:24:23.956740 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":89,"spinlock_wait_cycles":1408,"thread_start_us":129,"threads_started":1}
I20260811 22:24:23.959724 21696 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba2b806800c489ab70140201997c27a (DEFAULT_TABLE table=test-table [id=9ccbb057ad3a4e71810f1e3cf19dad96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:23.959728 21923 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba2b806800c489ab70140201997c27a (DEFAULT_TABLE table=test-table [id=9ccbb057ad3a4e71810f1e3cf19dad96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:23.960839 21923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba2b806800c489ab70140201997c27a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:23.960850 21696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba2b806800c489ab70140201997c27a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:23.962586 21447 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba2b806800c489ab70140201997c27a (DEFAULT_TABLE table=test-table [id=9ccbb057ad3a4e71810f1e3cf19dad96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:23.963833 21447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba2b806800c489ab70140201997c27a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:23.968034 22080 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: Bootstrap starting.
I20260811 22:24:23.969197 22081 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: Bootstrap starting.
I20260811 22:24:23.970664 22080 tablet_bootstrap.cc:654] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:23.971405 22082 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Bootstrap starting.
I20260811 22:24:23.971880 22081 tablet_bootstrap.cc:654] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:23.972081 22080 log.cc:824] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:23.972779 22081 log.cc:824] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:23.973822 22082 tablet_bootstrap.cc:654] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:23.974714 22082 log.cc:824] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:23.974821 22081 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: No bootstrap required, opened a new log
I20260811 22:24:23.975052 22080 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: No bootstrap required, opened a new log
I20260811 22:24:23.975072 22081 ts_tablet_manager.cc:1397] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:24:23.975286 22080 ts_tablet_manager.cc:1397] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:24:23.978034 22081 raft_consensus.cc:348] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:23.978309 22081 raft_consensus.cc:374] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:23.978420 22081 raft_consensus.cc:729] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a9ea189ef84652b6bbdd41ad8ecfaa, State: Initialized, Role: FOLLOWER
I20260811 22:24:23.978636 22080 raft_consensus.cc:348] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:23.978936 22080 raft_consensus.cc:374] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:23.979017 22080 raft_consensus.cc:729] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1977726fa0a94c7d85b35bc9d54ce61c, State: Initialized, Role: FOLLOWER
I20260811 22:24:23.978977 22081 consensus_queue.cc:260] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:23.979755 21819 heartbeater.cc:503] Master 127.26.125.62:44939 was elected leader, sending a full tablet report...
I20260811 22:24:23.980140 22081 ts_tablet_manager.cc:1428] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:24:23.980752 22082 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: No bootstrap required, opened a new log
I20260811 22:24:23.980983 22082 ts_tablet_manager.cc:1397] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260811 22:24:23.979539 22080 consensus_queue.cc:260] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:23.983357 22048 heartbeater.cc:503] Master 127.26.125.62:44939 was elected leader, sending a full tablet report...
I20260811 22:24:23.984535 22080 ts_tablet_manager.cc:1428] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260811 22:24:23.985188 22082 raft_consensus.cc:348] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:23.985492 22082 raft_consensus.cc:374] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:23.985556 22082 raft_consensus.cc:729] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8a171dfbb14cc38eb02d5506ae3456, State: Initialized, Role: FOLLOWER
I20260811 22:24:23.986039 22082 consensus_queue.cc:260] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:23.986765 21559 heartbeater.cc:503] Master 127.26.125.62:44939 was elected leader, sending a full tablet report...
I20260811 22:24:23.987197 22082 ts_tablet_manager.cc:1428] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:24:24.025674 22091 raft_consensus.cc:482] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:24.025928 22091 raft_consensus.cc:504] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:24.027215 22091 leader_election.cc:288] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803), 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
I20260811 22:24:24.031463 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:24.027624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:44351 (local address 127.26.125.2:40803)
0811 22:24:24.027769 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:24.027774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:24.027969 (+ 195us) server_negotiation.cc:407] Connection header received
0811 22:24:24.028170 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:24.028174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:24.028234 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:24.028315 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:24.028664 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.029304 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.030506 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.030934 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.031042 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:24.031100 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:24.031264 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 22:24:24.031334 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:24:24.031481 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:24.027518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:40803 (local address 127.26.125.1:44351)
0811 22:24:24.027861 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:24.027875 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:24.028038 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:24.028343 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:24.028346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:24.028374 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:24.028516 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:24.028523 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.029443 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.029447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:24.030377 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:24.030388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.031048 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.031051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:24.031149 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:24.031172 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:24:24.031241 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:24:24.031314 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":134,"threads_started":1}
I20260811 22:24:24.031738 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:24.028132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:33543 (local address 127.26.125.3:45463)
0811 22:24:24.028260 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:24.028264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:24.028402 (+ 138us) server_negotiation.cc:407] Connection header received
0811 22:24:24.028590 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:24.028593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:24.028642 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:24.028712 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:24.029037 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.029696 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.030747 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.031230 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.031339 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:24.031391 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:24.031563 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 22:24:24.031635 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:24:24.031750 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:24.027976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:45463 (local address 127.26.125.1:33543)
0811 22:24:24.028325 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:24.028337 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:24.028478 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:24.028720 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:24.028723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:24.028742 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:24.028865 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:24.028872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.029808 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.029812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:24.030608 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:24.030617 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.031341 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.031345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:24.031444 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:24.031469 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:24:24.031535 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:24:24.031602 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":118,"threads_started":1}
I20260811 22:24:24.032315 21730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba2b806800c489ab70140201997c27a" candidate_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" is_pre_election: true
I20260811 22:24:24.032513 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba2b806800c489ab70140201997c27a" candidate_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" is_pre_election: true
I20260811 22:24:24.032716 21730 raft_consensus.cc:2454] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8a171dfbb14cc38eb02d5506ae3456 in term 0.
I20260811 22:24:24.032757 21959 raft_consensus.cc:2454] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8a171dfbb14cc38eb02d5506ae3456 in term 0.
I20260811 22:24:24.033227 21364 leader_election.cc:302] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83a9ea189ef84652b6bbdd41ad8ecfaa, ba8a171dfbb14cc38eb02d5506ae3456; no voters:
I20260811 22:24:24.033506 22091 raft_consensus.cc:2781] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:24:24.033602 22091 raft_consensus.cc:482] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:24:24.033638 22091 raft_consensus.cc:3037] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:24.034727 22091 raft_consensus.cc:504] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:24.035113 22091 leader_election.cc:288] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 1 election: Requested vote from peers 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803), 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
I20260811 22:24:24.035487 21730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba2b806800c489ab70140201997c27a" candidate_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa"
I20260811 22:24:24.035588 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba2b806800c489ab70140201997c27a" candidate_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1977726fa0a94c7d85b35bc9d54ce61c"
I20260811 22:24:24.035633 21730 raft_consensus.cc:3037] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:24.035710 21959 raft_consensus.cc:3037] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:24.037003 21959 raft_consensus.cc:2454] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8a171dfbb14cc38eb02d5506ae3456 in term 1.
I20260811 22:24:24.037024 21730 raft_consensus.cc:2454] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8a171dfbb14cc38eb02d5506ae3456 in term 1.
I20260811 22:24:24.037320 21364 leader_election.cc:302] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83a9ea189ef84652b6bbdd41ad8ecfaa, ba8a171dfbb14cc38eb02d5506ae3456; no voters:
I20260811 22:24:24.037493 22091 raft_consensus.cc:2781] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:24.037763 22091 raft_consensus.cc:686] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 LEADER]: Becoming Leader. State: Replica: ba8a171dfbb14cc38eb02d5506ae3456, State: Running, Role: LEADER
I20260811 22:24:24.038149 22091 consensus_queue.cc:237] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:24.041407 21212 catalog_manager.cc:5495] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba8a171dfbb14cc38eb02d5506ae3456 (127.26.125.1). New cstate: current_term: 1 leader_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } health_report { overall_health: HEALTHY } } }
I20260811 22:24:24.056972 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:24.053276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.26.125.1:44389)
0811 22:24:24.053411 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:24.053415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:24.053430 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:24:24.053597 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:24.053600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:24.053658 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:24.053749 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:24.053992 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.054490 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.055451 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.055717 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.055809 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:24.056126 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:24.056568 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:24.056683 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:24.056761 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 22:24:24.056821 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:24:24.063261 21959 raft_consensus.cc:1261] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 1 FOLLOWER]: Refusing update from remote peer ba8a171dfbb14cc38eb02d5506ae3456: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:24.063275 21730 raft_consensus.cc:1261] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 FOLLOWER]: Refusing update from remote peer ba8a171dfbb14cc38eb02d5506ae3456: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:24.063952 22091 consensus_queue.cc:1035] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:24.064160 22101 consensus_queue.cc:1035] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:24.072517 22111 mvcc.cc:205] Tried to move back new op lower bound from 7317451014395392000 to 7317451014303305728. Current Snapshot: MvccSnapshot[applied={T|T < 7317451014395392000}]
I20260811 22:24:24.072579 22105 mvcc.cc:205] Tried to move back new op lower bound from 7317451014395392000 to 7317451014303305728. Current Snapshot: MvccSnapshot[applied={T|T < 7317451014395392000}]
I20260811 22:24:24.082101 21210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:24:24.082325 21210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:24:24.085410 21210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Sending DeleteTablet for 3 replicas of tablet 3ba2b806800c489ab70140201997c27a
I20260811 22:24:24.086405 21447 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba2b806800c489ab70140201997c27a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:24 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:58190
I20260811 22:24:24.086408 21923 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba2b806800c489ab70140201997c27a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:24 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:37496
I20260811 22:24:24.086673 21696 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba2b806800c489ab70140201997c27a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:24 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:43850
I20260811 22:24:24.087015 22126 tablet_replica.cc:330] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: stopping tablet replica
I20260811 22:24:24.087293 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 21326
I20260811 22:24:24.094470 22127 tablet_replica.cc:330] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: stopping tablet replica
I20260811 22:24:24.094900 22127 raft_consensus.cc:2229] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:24:24.095288 22134 tablet_replica.cc:330] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: stopping tablet replica
I20260811 22:24:24.095623 22127 raft_consensus.cc:2258] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:24.095641 22134 raft_consensus.cc:2229] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:24:24.095986 22134 raft_consensus.cc:2258] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:24.098672 22134 ts_tablet_manager.cc:1905] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:24.098681 22127 ts_tablet_manager.cc:1905] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:24.101030 22134 ts_tablet_manager.cc:1918] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:24.101138 22134 log.cc:1196] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/3ba2b806800c489ab70140201997c27a
I20260811 22:24:24.101444 22127 ts_tablet_manager.cc:1918] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:24.101496 22134 ts_tablet_manager.cc:1939] T 3ba2b806800c489ab70140201997c27a P 83a9ea189ef84652b6bbdd41ad8ecfaa: Deleting consensus metadata
I20260811 22:24:24.101567 22127 log.cc:1196] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/3ba2b806800c489ab70140201997c27a
I20260811 22:24:24.101938 22127 ts_tablet_manager.cc:1939] T 3ba2b806800c489ab70140201997c27a P 1977726fa0a94c7d85b35bc9d54ce61c: Deleting consensus metadata
I20260811 22:24:24.102980 21186 catalog_manager.cc:4841] TS 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803): tablet 3ba2b806800c489ab70140201997c27a (table test-table [id=9ccbb057ad3a4e71810f1e3cf19dad96]) successfully deleted
I20260811 22:24:24.103221 21186 catalog_manager.cc:4841] TS 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463): tablet 3ba2b806800c489ab70140201997c27a (table test-table [id=9ccbb057ad3a4e71810f1e3cf19dad96]) successfully deleted
W20260811 22:24:24.105121 21177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:44389 (error 108)
I20260811 22:24:24.105358 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 21567
I20260811 22:24:24.108726 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:24.108327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:44389 (local address 127.0.0.1:58204)
0811 22:24:24.108539 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:24.108622 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260811 22:24:24.108950 21177 catalog_manager.cc:4584] TS ba8a171dfbb14cc38eb02d5506ae3456 (127.26.125.1:44389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ba2b806800c489ab70140201997c27a: Network error: Client connection negotiation failed: client connection to 127.26.125.1:44389: connect: Connection refused (error 111)
I20260811 22:24:24.117044 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 21828
I20260811 22:24:24.133124 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 21150
I20260811 22:24:44.155264 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44939
--webserver_interface=127.26.125.62
--webserver_port=37207
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:44.282264 22138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:44.282765 22138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:44.282871 22138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:44.287842 22138 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:24:44.287947 22138 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:24:44.287973 22138 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:24:44.287992 22138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:44.288065 22138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:44.288105 22138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:44.288129 22138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:44.288158 22138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:44.296109 22138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=37207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:44.297655 22138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:44.298939 22138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:44.317934 22149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:44.317935 22151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:44.318305 22138 server_base.cc:1034] running on GCE node
W20260811 22:24:44.317934 22146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:44.319273 22138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:44.321002 22138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:44.322259 22138 hybrid_clock.cc:648] HybridClock initialized: now 1786487084322219 us; error 60 us; skew 500 ppm
Aug 11 22:24:44 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:44.326252 22138 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:24:44.327785 22138 webserver.cc:460] Webserver started at http://127.26.125.62:37207/ using document root <none> and password file <none>
I20260811 22:24:44.328648 22138 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:44.328768 22138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:44.333585 22138 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:24:44.336040 22158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:44.337502 22138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:44.337630 22138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d8ed4d8f42e51c6fa808db4be85985c7"
server_key_iv: "bdbbc4f437dc70b6016184fdd8063d31"
server_key_version: "encryptionkey@0"
I20260811 22:24:44.338243 22138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:44.366098 22138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:44.369908 22138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:44.370139 22138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:44.377718 22138 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:44939
I20260811 22:24:44.377980 22252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:44939 every 8 connection(s)
I20260811 22:24:44.378940 22138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:44.383744 22254 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:24:44.385511 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Bootstrap starting.
I20260811 22:24:44.387343 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 22138
I20260811 22:24:44.387856 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:44389
--local_ip_for_outbound_sockets=127.26.125.1
--tserver_master_addrs=127.26.125.62:44939
--webserver_port=33159
--webserver_interface=127.26.125.1
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:44.403882 22254 log.cc:824] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:44.414566 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:24:44.415030 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Bootstrap complete.
I20260811 22:24:44.419216 22254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:24:44.419698 22254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c45d0b5bf7e4519b5c047b7601b9d5b, State: Initialized, Role: FOLLOWER
I20260811 22:24:44.420302 22254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:24:44.420446 22254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:44.420531 22254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:44.420634 22254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:24:44.422900 22254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:24:44.423306 22254 leader_election.cc:302] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c45d0b5bf7e4519b5c047b7601b9d5b; no voters:
I20260811 22:24:44.423695 22254 leader_election.cc:288] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:24:44.423808 22260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:24:44.424026 22260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 2 LEADER]: Becoming Leader. State: Replica: 6c45d0b5bf7e4519b5c047b7601b9d5b, State: Running, Role: LEADER
I20260811 22:24:44.424360 22260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:24:44.425220 22254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:44.425912 22261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } } }
I20260811 22:24:44.426072 22261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:44.426553 22263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c45d0b5bf7e4519b5c047b7601b9d5b. Latest consensus state: current_term: 2 leader_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } } }
I20260811 22:24:44.426703 22263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:44.427115 22273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:44.432619 22273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:44.433418 22273 catalog_manager.cc:1259] Loaded cluster ID: 67c27cc5383640f986db59de45b42ca3
I20260811 22:24:44.433547 22273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:44.436196 22273 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:44.437021 22273 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Loaded TSK: 0
I20260811 22:24:44.452165 22273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Generated new TSK 1
I20260811 22:24:44.452625 22273 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:24:44.522246 22258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:44.522583 22258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:44.522646 22258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:44.527118 22258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:44.527200 22258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:44.527287 22258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:24:44.533013 22258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:44389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=33159
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:44.534188 22258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:44.535290 22258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:44.545728 22301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:44.545739 22298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:44.545799 22306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:44.546195 22258 server_base.cc:1034] running on GCE node
I20260811 22:24:44.546667 22258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:44.547446 22258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:44.548642 22258 hybrid_clock.cc:648] HybridClock initialized: now 1786487084548603 us; error 51 us; skew 500 ppm
Aug 11 22:24:44 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:44.552024 22258 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:24:44.553126 22258 webserver.cc:460] Webserver started at http://127.26.125.1:33159/ using document root <none> and password file <none>
I20260811 22:24:44.553750 22258 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:44.553846 22258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:44.558213 22258 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:44.560706 22317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:44.562115 22258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:24:44.562248 22258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "ba8a171dfbb14cc38eb02d5506ae3456"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "cf0c991e6fabf2921d7af78f72506b07"
server_key_iv: "eee5f2e81296a1a23ea935cf52083dbe"
server_key_version: "encryptionkey@0"
I20260811 22:24:44.562831 22258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:44.582460 22258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:44.586078 22258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:44.586321 22258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:44.587090 22258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:44.588406 22331 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:24:44.593012 22258 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:24:44.593113 22258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:24:44.593194 22258 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:24:44.595090 22258 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:24:44.595152 22258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 22:24:44.595461 22331 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Bootstrap starting.
I20260811 22:24:44.605638 22258 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:44389
I20260811 22:24:44.605719 22531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:44389 every 8 connection(s)
I20260811 22:24:44.606695 22258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:24:44.614005 22331 log.cc:824] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:44.616875 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 22258
I20260811 22:24:44.617390 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:40803
--local_ip_for_outbound_sockets=127.26.125.2
--tserver_master_addrs=127.26.125.62:44939
--webserver_port=34961
--webserver_interface=127.26.125.2
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:24:44 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:44.622659 22331 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:24:44.623163 22331 tablet_bootstrap.cc:492] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Bootstrap complete.
I20260811 22:24:44.624394 22331 ts_tablet_manager.cc:1397] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.018s
I20260811 22:24:44.625365 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:44.609003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:54927)
0811 22:24:44.609608 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:44.609655 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:44.610565 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:44.611844 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:44.611853 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:44.612318 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:44.612807 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:44.612821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.614140 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.614143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:44.615109 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:44.615120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.615874 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.615878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:44.616066 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:44.616758 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:44.616780 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:44.618662 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:44.621431 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:44.621438 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:44.621454 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:44.621813 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:44.622261 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:44.622264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:44.622266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:44.622474 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:44.623381 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:44.623388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:44.623698 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 22:24:44.623907 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 22:24:44.624283 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":191,"threads_started":1}
I20260811 22:24:44.625602 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:44.608933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:54927 (local address 127.26.125.62:44939)
0811 22:24:44.609309 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:44.609327 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:44.609797 (+ 470us) server_negotiation.cc:407] Connection header received
0811 22:24:44.611077 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:44.611090 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:44.611418 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:44.611757 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:44.612996 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.613985 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.615227 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.615732 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.615895 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:44.618934 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:44.618942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:44.618961 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:44.619010 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:44.621273 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:44.621968 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:44.621975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:44.621981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:44.622152 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:44.622613 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:44.622616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:44.622618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:44.623264 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:44.623428 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:44.624072 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 22:24:44.624439 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":105,"threads_started":1}
I20260811 22:24:44.627455 22331 raft_consensus.cc:348] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:44.628005 22331 raft_consensus.cc:729] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8a171dfbb14cc38eb02d5506ae3456, State: Initialized, Role: FOLLOWER
I20260811 22:24:44.628362 22532 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44939
I20260811 22:24:44.628572 22331 consensus_queue.cc:260] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:44.628640 22532 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:44.629308 22532 heartbeater.cc:511] Master 127.26.125.62:44939 requested a full tablet report, sending...
I20260811 22:24:44.629603 22331 ts_tablet_manager.cc:1428] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 22:24:44.632278 22200 ts_manager.cc:194] Registered new tserver with Master: ba8a171dfbb14cc38eb02d5506ae3456 (127.26.125.1:44389)
I20260811 22:24:44.635916 22200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:54927
I20260811 22:24:44.638644 22532 heartbeater.cc:503] Master 127.26.125.62:44939 was elected leader, sending a full tablet report...
Aug 11 22:24:44 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.1@KRBTEST.COM
I20260811 22:24:44.646943 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:44.635401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:44389 (local address 127.0.0.1:33682)
0811 22:24:44.636233 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:44.636279 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:44.636437 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:44.637017 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:44.637024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:44.637507 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:44.637646 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:44.637656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.639076 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.639080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:44.639934 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:44.639947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.640749 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.640751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:44.640857 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:44.641446 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:44.641459 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:44.643196 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:44.645247 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:44.645256 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:44.645263 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:44.645502 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:44.645799 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:44.645801 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:44.645802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:44.645878 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:44.646359 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:44.646364 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:44.646477 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 22:24:44.646590 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:24:44.646753 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":118,"threads_started":1}
I20260811 22:24:44.647065 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:44.635508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.26.125.1:44389)
0811 22:24:44.635946 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:44.635959 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:44.636399 (+ 440us) server_negotiation.cc:407] Connection header received
0811 22:24:44.636613 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:44.636620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:44.636843 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:44.636979 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:44.637769 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.638964 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.640086 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.640624 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.640719 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:44.643363 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:44.643375 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:44.643390 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:44.643428 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:44.645119 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:44.645610 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:44.645616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:44.645618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:44.645715 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:44.645948 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:44.645951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:44.645952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:44.646261 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:44.646391 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:44.646687 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:24:44.646893 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":137,"threads_started":1}
I20260811 22:24:44.648137 22402 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba2b806800c489ab70140201997c27a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:24 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:33682
I20260811 22:24:44.649195 22546 tablet_replica.cc:330] stopping tablet replica
I20260811 22:24:44.649567 22546 raft_consensus.cc:2229] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:24:44.649828 22546 raft_consensus.cc:2258] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:44.650964 22546 ts_tablet_manager.cc:1905] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:44.654963 22546 ts_tablet_manager.cc:1918] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:44.655072 22546 log.cc:1196] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal/wals/3ba2b806800c489ab70140201997c27a
I20260811 22:24:44.655304 22546 ts_tablet_manager.cc:1939] T 3ba2b806800c489ab70140201997c27a P ba8a171dfbb14cc38eb02d5506ae3456: Deleting consensus metadata
I20260811 22:24:44.657541 22160 catalog_manager.cc:4841] TS ba8a171dfbb14cc38eb02d5506ae3456 (127.26.125.1:44389): tablet 3ba2b806800c489ab70140201997c27a (table test-table [id=9ccbb057ad3a4e71810f1e3cf19dad96]) successfully deleted
W20260811 22:24:44.750178 22539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:44.750566 22539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:44.750643 22539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:44.754951 22539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:44.755033 22539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:44.755120 22539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:24:44.760777 22539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:40803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=34961
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:44.762018 22539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:44.763377 22539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:44.774598 22563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:44.774696 22558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:44.774708 22560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:44.775032 22539 server_base.cc:1034] running on GCE node
I20260811 22:24:44.775538 22539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:44.776279 22539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:44.777449 22539 hybrid_clock.cc:648] HybridClock initialized: now 1786487084777428 us; error 33 us; skew 500 ppm
Aug 11 22:24:44 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:44.780328 22539 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:24:44.781292 22539 webserver.cc:460] Webserver started at http://127.26.125.2:34961/ using document root <none> and password file <none>
I20260811 22:24:44.781946 22539 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:44.782043 22539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:44.786161 22539 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:44.788658 22579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:44.790220 22539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:24:44.790351 22539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "c8976bed2865d2be27c73254d2c4f8a8"
server_key_iv: "135b6a40f47d156d4398a789b39bdd27"
server_key_version: "encryptionkey@0"
I20260811 22:24:44.791050 22539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:44.812955 22539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:44.816269 22539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:44.816488 22539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:44.817147 22539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:44.818053 22539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:44.818101 22539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:44.818148 22539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:44.818176 22539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:44.827739 22539 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:40803
I20260811 22:24:44.827760 22776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:40803 every 8 connection(s)
I20260811 22:24:44.828845 22539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:24:44.837009 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 22539
I20260811 22:24:44.837415 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:45463
--local_ip_for_outbound_sockets=127.26.125.3
--tserver_master_addrs=127.26.125.62:44939
--webserver_port=40395
--webserver_interface=127.26.125.3
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:24:44 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:44.842566 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:44.830822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:43569 (local address 127.26.125.62:44939)
0811 22:24:44.830995 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:44.830999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:44.832138 (+ 1139us) server_negotiation.cc:407] Connection header received
0811 22:24:44.833005 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:44.833008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:44.833058 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:44.833139 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:44.834316 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.834920 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.836112 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.836343 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.836432 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:44.838925 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:44.838928 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:44.838930 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:44.838956 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:44.840297 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:44.840778 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:44.840781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:44.840782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:44.840859 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:44.841221 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:44.841225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:44.841227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:44.841499 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:44.841621 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:44.842229 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 22:24:44.842377 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:24:44.843466 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:44.831319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:43569)
0811 22:24:44.831978 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:44.832031 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:44.832802 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:44.833295 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:44.833302 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:44.833678 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:44.834181 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:44.834195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.835048 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.835050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:44.836007 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:44.836017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:44.836464 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:44.836467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:44.836605 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:44.837213 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:44.837229 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:44.838728 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:44.840410 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:44.840417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:44.840433 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:44.840684 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:44.840944 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:44.840947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:44.840949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:44.841086 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:44.841644 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:44.841649 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:44.841943 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 22:24:44.842155 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 22:24:44.842467 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":117,"threads_started":1}
I20260811 22:24:44.844652 22777 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44939
I20260811 22:24:44.844930 22777 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:44.845563 22777 heartbeater.cc:511] Master 127.26.125.62:44939 requested a full tablet report, sending...
I20260811 22:24:44.846800 22200 ts_manager.cc:194] Registered new tserver with Master: 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803)
I20260811 22:24:44.847430 22200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:43569
W20260811 22:24:44.955886 22783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:44.956157 22783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:44.956204 22783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:44.960216 22783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:44.960273 22783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:44.960337 22783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:24:44.965929 22783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:45463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=40395
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:24:44.967135 22783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:44.968250 22783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:44.978590 22793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:44.978606 22790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:44.978624 22796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:44.978826 22783 server_base.cc:1034] running on GCE node
I20260811 22:24:44.979391 22783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:44.980140 22783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:44.981330 22783 hybrid_clock.cc:648] HybridClock initialized: now 1786487084981303 us; error 43 us; skew 500 ppm
Aug 11 22:24:44 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:44.987702 22783 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:24:44.988933 22783 webserver.cc:460] Webserver started at http://127.26.125.3:40395/ using document root <none> and password file <none>
I20260811 22:24:44.989626 22783 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:44.989734 22783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:44.994375 22783 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:44.996973 22810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:44.998555 22783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:44.998703 22783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "1977726fa0a94c7d85b35bc9d54ce61c"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d57245a3360de3eaafce5f80c42edb99"
server_key_iv: "6b980bd91dfddc0b9f3ee7214740d059"
server_key_version: "encryptionkey@0"
I20260811 22:24:44.999384 22783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:45.018141 22783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:45.021628 22783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:45.021832 22783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:45.022505 22783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:24:45.023474 22783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:24:45.023537 22783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:45.023597 22783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:24:45.023622 22783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:45.032721 22783 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:45463
I20260811 22:24:45.032750 23007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:45463 every 8 connection(s)
I20260811 22:24:45.033799 22783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:24:45.035485 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 22783
Aug 11 22:24:45 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:24:45.044723 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.036095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.26.125.62:44939)
0811 22:24:45.036336 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.036340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.036498 (+ 158us) server_negotiation.cc:407] Connection header received
0811 22:24:45.036622 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.036625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.036679 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.036778 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:45.037705 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.038280 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.039517 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.039750 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.039839 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.041472 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:45.041475 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:45.041477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:45.041500 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:45.043098 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:45.043493 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:45.043497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:45.043498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:45.043580 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:45.043836 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:45.043839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:45.043841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:45.044027 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:45.044237 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.044420 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.044550 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":96,"threads_started":1}
I20260811 22:24:45.048027 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.035746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:47221 (local address 127.26.125.62:44939)
0811 22:24:45.035899 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.035904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.036907 (+ 1003us) server_negotiation.cc:407] Connection header received
0811 22:24:45.038415 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.038418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.038458 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.038507 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:45.040013 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.040487 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.041965 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.042287 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.042387 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.044815 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:45.044819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:45.044821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:45.044845 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:45.046124 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:45.046653 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:45.046661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:45.046662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:45.046737 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:45.047094 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:45.047097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:45.047098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:45.047263 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:45.047338 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.047806 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.047902 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:24:45.048987 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.036237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:47221)
0811 22:24:45.036770 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:45.036827 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:45.038200 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:45.038704 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:45.038713 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:45.039230 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:45.039847 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.039867 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.040640 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.040643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.041848 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.041858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.042474 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.042480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.042665 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.043134 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:45.043151 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:45.044642 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:45.046229 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:45.046235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:45.046253 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:45.046568 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:45.046813 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:45.046815 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:45.046816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:45.046995 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:45.047339 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:45.047344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:45.047599 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 22:24:45.047756 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:24:45.048051 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":130,"threads_started":1}
I20260811 22:24:45.050230 23010 heartbeater.cc:347] Connected to a master server at 127.26.125.62:44939
I20260811 22:24:45.050521 23010 heartbeater.cc:464] Registering TS with master...
I20260811 22:24:45.051157 23010 heartbeater.cc:511] Master 127.26.125.62:44939 requested a full tablet report, sending...
I20260811 22:24:45.052273 22200 ts_manager.cc:194] Registered new tserver with Master: 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
I20260811 22:24:45.052868 22200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:47221
I20260811 22:24:45.058404 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:24:45.062480 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.059084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57166 (local address 127.26.125.62:44939)
0811 22:24:45.059221 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.059226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.059405 (+ 179us) server_negotiation.cc:407] Connection header received
0811 22:24:45.059517 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.059519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.059625 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.059714 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:45.059953 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.060460 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.061339 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.061565 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.061652 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.061958 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:45.062112 (+ 154us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 22:24:45.062337 (+ 225us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57166: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 22:24:45.062577 22535 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57166: token expired
W20260811 22:24:45.063004 22058 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.26.125.62:44939)
I20260811 22:24:45.070242 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.063364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.26.125.62:44939)
0811 22:24:45.063479 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.063483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.063495 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:24:45.063602 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.063605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.063647 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.063709 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:45.064347 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.064846 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.065635 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.065841 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.065924 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.067249 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:45.067252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:45.067254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:45.067279 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:45.068628 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:45.069031 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:45.069035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:45.069037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:45.069118 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:45.069381 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:45.069384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:45.069386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:45.069587 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:45.069708 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.069949 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.070065 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:24:45.073010 22200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:24:45.076491 22200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:24:45.085770 22402 tablet_service.cc:1467] Processing CreateTablet for tablet 75b73ed531ea44dc91dc887be4215382 (DEFAULT_TABLE table=test-table [id=1fedc7ee03b747858f86581480390b3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:45.086592 22402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b73ed531ea44dc91dc887be4215382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:45.089813 22331 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Bootstrap starting.
I20260811 22:24:45.090517 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.084627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:40803 (local address 127.0.0.1:37174)
0811 22:24:45.084758 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:45.084776 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:45.084869 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:45.086093 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:45.086096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:45.086115 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:45.086258 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.086266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.087942 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.087947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.088895 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.088907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.090009 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.090013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.090138 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.090212 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 22:24:45.090269 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 22:24:45.090363 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":41,"spinlock_wait_cycles":8960}
I20260811 22:24:45.090826 22331 tablet_bootstrap.cc:654] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:45.091101 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.084873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:45463 (local address 127.0.0.1:51652)
0811 22:24:45.085168 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:45.085182 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:45.085270 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:45.086141 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:45.086145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:45.086166 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:45.086289 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.086295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.088033 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.088037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.089139 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.089151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.090706 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.090710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.090793 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.090816 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:24:45.090896 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:24:45.090969 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":1,"thread_start_us":142,"threads_started":1}
I20260811 22:24:45.091248 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.084808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.26.125.2:40803)
0811 22:24:45.085506 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.085520 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.085547 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:24:45.085634 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.085640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.085896 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.086433 (+ 537us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:45.086636 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.087775 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.089032 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.089847 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.090721 (+ 874us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.090943 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.091003 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.091107 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":200,"threads_started":1}
I20260811 22:24:45.091887 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.085376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.26.125.3:45463)
0811 22:24:45.085702 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.085714 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.085736 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:24:45.085803 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.085807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.085973 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.086475 (+ 502us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:45.086660 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.087840 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.089550 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.090547 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.091287 (+ 740us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.091586 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.091644 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.091745 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":118,"threads_started":1}
I20260811 22:24:45.092644 22331 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: No bootstrap required, opened a new log
I20260811 22:24:45.092756 22331 ts_tablet_manager.cc:1397] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:45.093259 22331 raft_consensus.cc:348] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.093375 22331 raft_consensus.cc:374] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:45.093410 22331 raft_consensus.cc:729] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8a171dfbb14cc38eb02d5506ae3456, State: Initialized, Role: FOLLOWER
I20260811 22:24:45.093544 22331 consensus_queue.cc:260] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.093802 22331 ts_tablet_manager.cc:1428] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:24:45.094830 22888 tablet_service.cc:1467] Processing CreateTablet for tablet 75b73ed531ea44dc91dc887be4215382 (DEFAULT_TABLE table=test-table [id=1fedc7ee03b747858f86581480390b3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:45.094828 22671 tablet_service.cc:1467] Processing CreateTablet for tablet 75b73ed531ea44dc91dc887be4215382 (DEFAULT_TABLE table=test-table [id=1fedc7ee03b747858f86581480390b3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:24:45.095901 22671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b73ed531ea44dc91dc887be4215382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:45.095917 22888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b73ed531ea44dc91dc887be4215382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:45.103230 23030 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Bootstrap starting.
I20260811 22:24:45.103740 23031 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Bootstrap starting.
I20260811 22:24:45.105798 23031 tablet_bootstrap.cc:654] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:45.106153 23030 tablet_bootstrap.cc:654] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:45.106842 23031 log.cc:824] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:45.106842 23030 log.cc:824] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Log is configured to *not* fsync() on all Append() calls
W20260811 22:24:45.108011 22533 tablet.cc:2369] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:45.108803 23031 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: No bootstrap required, opened a new log
I20260811 22:24:45.108810 23030 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: No bootstrap required, opened a new log
I20260811 22:24:45.108984 23031 ts_tablet_manager.cc:1397] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:24:45.108985 23030 ts_tablet_manager.cc:1397] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:24:45.112219 23030 raft_consensus.cc:348] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.112219 23031 raft_consensus.cc:348] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.112474 23030 raft_consensus.cc:374] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:45.112481 23031 raft_consensus.cc:374] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:45.112514 23030 raft_consensus.cc:729] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a9ea189ef84652b6bbdd41ad8ecfaa, State: Initialized, Role: FOLLOWER
I20260811 22:24:45.112547 23031 raft_consensus.cc:729] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1977726fa0a94c7d85b35bc9d54ce61c, State: Initialized, Role: FOLLOWER
I20260811 22:24:45.112948 23031 consensus_queue.cc:260] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.112947 23030 consensus_queue.cc:260] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.113656 22777 heartbeater.cc:503] Master 127.26.125.62:44939 was elected leader, sending a full tablet report...
I20260811 22:24:45.113970 23031 ts_tablet_manager.cc:1428] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:24:45.114120 23010 heartbeater.cc:503] Master 127.26.125.62:44939 was elected leader, sending a full tablet report...
I20260811 22:24:45.114789 23030 ts_tablet_manager.cc:1428] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:24:45.282783 23035 raft_consensus.cc:482] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:24:45.283083 23035 raft_consensus.cc:504] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.284397 23035 leader_election.cc:288] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463), ba8a171dfbb14cc38eb02d5506ae3456 (127.26.125.1:44389)
W20260811 22:24:45.284792 23011 tablet.cc:2369] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:45.288582 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.284748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:45463 (local address 127.26.125.2:57329)
0811 22:24:45.285133 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:45.285150 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:45.285261 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:45.285580 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:45.285583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:45.285606 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:45.285736 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.285744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.286540 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.286543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.287454 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.287465 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.288118 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.288122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.288243 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.288272 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:24:45.288341 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:24:45.288419 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":65}
I20260811 22:24:45.288611 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.285001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:57329 (local address 127.26.125.3:45463)
0811 22:24:45.285244 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.285249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.285270 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:24:45.285409 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.285412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.285459 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.285517 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:45.285843 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.286420 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.287589 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.288021 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.288117 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.288162 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.288391 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.288480 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 22:24:45.288853 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.284748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:44389 (local address 127.26.125.2:36429)
0811 22:24:45.285243 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:45.285255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:45.285332 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:45.285795 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:45.285798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:45.285818 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:24:45.285938 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.285945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.286699 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.286701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.287527 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:45.287540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.288357 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.288361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:45.288533 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.288559 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:24:45.288645 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:24:45.288718 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":23,"thread_start_us":162,"threads_started":1}
I20260811 22:24:45.289156 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.284965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:36429 (local address 127.26.125.1:44389)
0811 22:24:45.285507 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.285512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.285527 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:24:45.285578 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.285581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.285642 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.285773 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:24:45.286073 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.286595 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.287660 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.288206 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.288343 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.288503 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.288949 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.289048 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":105,"threads_started":1}
I20260811 22:24:45.289327 22915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b73ed531ea44dc91dc887be4215382" candidate_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" is_pre_election: true
I20260811 22:24:45.289690 22915 raft_consensus.cc:2454] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83a9ea189ef84652b6bbdd41ad8ecfaa in term 0.
I20260811 22:24:45.289674 22448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b73ed531ea44dc91dc887be4215382" candidate_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" is_pre_election: true
I20260811 22:24:45.289934 22448 raft_consensus.cc:2454] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83a9ea189ef84652b6bbdd41ad8ecfaa in term 0.
I20260811 22:24:45.290226 22584 leader_election.cc:302] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1977726fa0a94c7d85b35bc9d54ce61c, 83a9ea189ef84652b6bbdd41ad8ecfaa; no voters:
I20260811 22:24:45.290529 23035 raft_consensus.cc:2781] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:24:45.290614 23035 raft_consensus.cc:482] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:24:45.290645 23035 raft_consensus.cc:3037] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:45.291771 23035 raft_consensus.cc:504] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.292145 23035 leader_election.cc:288] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [CANDIDATE]: Term 1 election: Requested vote from peers 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463), ba8a171dfbb14cc38eb02d5506ae3456 (127.26.125.1:44389)
I20260811 22:24:45.292531 22448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b73ed531ea44dc91dc887be4215382" candidate_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8a171dfbb14cc38eb02d5506ae3456"
I20260811 22:24:45.292680 22448 raft_consensus.cc:3037] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:45.293694 22448 raft_consensus.cc:2454] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83a9ea189ef84652b6bbdd41ad8ecfaa in term 1.
I20260811 22:24:45.293941 22915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b73ed531ea44dc91dc887be4215382" candidate_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1977726fa0a94c7d85b35bc9d54ce61c"
I20260811 22:24:45.294004 22580 leader_election.cc:302] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83a9ea189ef84652b6bbdd41ad8ecfaa, ba8a171dfbb14cc38eb02d5506ae3456; no voters:
I20260811 22:24:45.294060 22915 raft_consensus.cc:3037] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:45.294200 23035 raft_consensus.cc:2781] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:45.294458 23035 raft_consensus.cc:686] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 LEADER]: Becoming Leader. State: Replica: 83a9ea189ef84652b6bbdd41ad8ecfaa, State: Running, Role: LEADER
I20260811 22:24:45.294888 23035 consensus_queue.cc:237] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:24:45.295539 22915 raft_consensus.cc:2454] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83a9ea189ef84652b6bbdd41ad8ecfaa in term 1.
I20260811 22:24:45.298132 22199 catalog_manager.cc:5495] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2). New cstate: current_term: 1 leader_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } health_report { overall_health: UNKNOWN } } }
W20260811 22:24:45.330113 22780 tablet.cc:2369] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:45.331498 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.327537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.26.125.2:40803)
0811 22:24:45.327681 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.327685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.327702 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:24:45.327845 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.327848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.327908 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.328012 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:45.328262 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.328773 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.329676 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:45.329933 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:45.330045 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:45.330494 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:45.331068 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:45.331171 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.331278 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.331339 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:24:45.338366 22448 raft_consensus.cc:1261] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Refusing update from remote peer 83a9ea189ef84652b6bbdd41ad8ecfaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:45.338371 22915 raft_consensus.cc:1261] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 1 FOLLOWER]: Refusing update from remote peer 83a9ea189ef84652b6bbdd41ad8ecfaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:45.339162 23041 consensus_queue.cc:1035] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:45.339352 23035 consensus_queue.cc:1035] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:45.346822 23046 mvcc.cc:205] Tried to move back new op lower bound from 7317451101537251328 to 7317451101371367424. Current Snapshot: MvccSnapshot[applied={T|T < 7317451101537251328 or (T in {7317451101537251328})}]
I20260811 22:24:45.348253 23048 mvcc.cc:205] Tried to move back new op lower bound from 7317451101537251328 to 7317451101371367424. Current Snapshot: MvccSnapshot[applied={T|T < 7317451101537251328}]
I20260811 22:24:45.356786 22199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:24:45.356974 22199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:24:45.359884 22199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Sending DeleteTablet for 3 replicas of tablet 75b73ed531ea44dc91dc887be4215382
I20260811 22:24:45.360819 22402 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b73ed531ea44dc91dc887be4215382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:45 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:33682
I20260811 22:24:45.360919 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 22258
I20260811 22:24:45.361073 22671 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b73ed531ea44dc91dc887be4215382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:45 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:37174
I20260811 22:24:45.361678 22888 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b73ed531ea44dc91dc887be4215382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:24:45 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:51652
I20260811 22:24:45.366238 23060 tablet_replica.cc:330] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: stopping tablet replica
I20260811 22:24:45.366736 23060 raft_consensus.cc:2229] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:24:45.367353 23060 raft_consensus.cc:2258] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:45.371091 23060 ts_tablet_manager.cc:1905] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:45.375391 23060 ts_tablet_manager.cc:1918] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:45.375806 23065 tablet_replica.cc:330] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: stopping tablet replica
I20260811 22:24:45.376015 23060 log.cc:1196] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/75b73ed531ea44dc91dc887be4215382
I20260811 22:24:45.376484 23065 raft_consensus.cc:2229] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:24:45.376961 23060 ts_tablet_manager.cc:1939] T 75b73ed531ea44dc91dc887be4215382 P 83a9ea189ef84652b6bbdd41ad8ecfaa: Deleting consensus metadata
I20260811 22:24:45.377416 23065 raft_consensus.cc:2258] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:24:45.378984 22164 catalog_manager.cc:4841] TS 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803): tablet 75b73ed531ea44dc91dc887be4215382 (table test-table [id=1fedc7ee03b747858f86581480390b3b]) successfully deleted
I20260811 22:24:45.382454 23065 ts_tablet_manager.cc:1905] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:24:45.385777 23065 ts_tablet_manager.cc:1918] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:24:45.385887 23065 log.cc:1196] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/75b73ed531ea44dc91dc887be4215382
I20260811 22:24:45.386209 23065 ts_tablet_manager.cc:1939] T 75b73ed531ea44dc91dc887be4215382 P 1977726fa0a94c7d85b35bc9d54ce61c: Deleting consensus metadata
I20260811 22:24:45.387632 22164 catalog_manager.cc:4841] TS 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463): tablet 75b73ed531ea44dc91dc887be4215382 (table test-table [id=1fedc7ee03b747858f86581480390b3b]) successfully deleted
W20260811 22:24:45.388618 22160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:44389 (error 108)
W20260811 22:24:45.390033 23025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:45.389732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:44389 (local address 127.0.0.1:33686)
0811 22:24:45.389875 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:45.389947 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:44389: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 22:24:45.390319 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 22539
W20260811 22:24:45.390336 22160 catalog_manager.cc:4584] TS ba8a171dfbb14cc38eb02d5506ae3456 (127.26.125.1:44389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75b73ed531ea44dc91dc887be4215382: Network error: Client connection negotiation failed: client connection to 127.26.125.1:44389: connect: Connection reset by peer (error 104)
I20260811 22:24:45.403728 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 22783
I20260811 22:24:45.426636 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 22138
I20260811 22:25:05.441267 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44939
--webserver_interface=127.26.125.62
--webserver_port=37207
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:05.566329 23070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:05.566640 23070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:05.566699 23070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:05.570869 23070 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:25:05.570945 23070 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:25:05.570966 23070 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:25:05.570983 23070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:05.571007 23070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:05.571031 23070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:05.571060 23070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:05.571089 23070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:05.576879 23070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=37207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:05.578099 23070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:05.579245 23070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:05.588675 23070 server_base.cc:1034] running on GCE node
W20260811 22:25:05.588626 23077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:05.588693 23078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:05.588626 23082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:05.589560 23070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:05.590740 23070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:05.591923 23070 hybrid_clock.cc:648] HybridClock initialized: now 1786487105591883 us; error 48 us; skew 500 ppm
Aug 11 22:25:05 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487105, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:05.594823 23070 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:05.595834 23070 webserver.cc:460] Webserver started at http://127.26.125.62:37207/ using document root <none> and password file <none>
I20260811 22:25:05.596392 23070 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:05.596454 23070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:05.600829 23070 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:25:05.603555 23094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:05.605140 23070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:05.605298 23070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d8ed4d8f42e51c6fa808db4be85985c7"
server_key_iv: "bdbbc4f437dc70b6016184fdd8063d31"
server_key_version: "encryptionkey@0"
I20260811 22:25:05.605912 23070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:05.627202 23070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:05.630520 23070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:05.630707 23070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:05.638504 23070 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:44939
I20260811 22:25:05.638803 23192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:44939 every 8 connection(s)
I20260811 22:25:05.639783 23070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:05.640573 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 23070
I20260811 22:25:05.641103 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:44389
--local_ip_for_outbound_sockets=127.26.125.1
--tserver_master_addrs=127.26.125.62:44939
--webserver_port=33159
--webserver_interface=127.26.125.1
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:25:05.645107 23194 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:25:05.646960 23194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Bootstrap starting.
I20260811 22:25:05.664526 23194 log.cc:824] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:05.676451 23194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:25:05.676856 23194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Bootstrap complete.
I20260811 22:25:05.680874 23194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:25:05.681306 23194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c45d0b5bf7e4519b5c047b7601b9d5b, State: Initialized, Role: FOLLOWER
I20260811 22:25:05.681829 23194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:25:05.681951 23194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:05.682020 23194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:05.682121 23194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:25:05.684568 23194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:25:05.684909 23194 leader_election.cc:302] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c45d0b5bf7e4519b5c047b7601b9d5b; no voters:
I20260811 22:25:05.685231 23194 leader_election.cc:288] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:25:05.685335 23201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:25:05.685549 23201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [term 3 LEADER]: Becoming Leader. State: Replica: 6c45d0b5bf7e4519b5c047b7601b9d5b, State: Running, Role: LEADER
I20260811 22:25:05.685927 23201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } }
I20260811 22:25:05.686604 23194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:05.687148 23201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c45d0b5bf7e4519b5c047b7601b9d5b. Latest consensus state: current_term: 3 leader_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } } }
I20260811 22:25:05.687351 23201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:05.687518 23204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c45d0b5bf7e4519b5c047b7601b9d5b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 44939 } } }
I20260811 22:25:05.687644 23204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:05.688120 23210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:05.693775 23210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:05.694602 23210 catalog_manager.cc:1259] Loaded cluster ID: 67c27cc5383640f986db59de45b42ca3
I20260811 22:25:05.694679 23210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:05.697338 23210 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:05.698167 23210 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Loaded TSK: 0
I20260811 22:25:05.698261 23210 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c45d0b5bf7e4519b5c047b7601b9d5b: Loaded TSK: 1
F20260811 22:25:05.699087 23210 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f44ab0039ad google::LogMessage::Fail() at ??:0
@ 0x7f44ab00688a google::LogMessage::SendToLog() at ??:0
@ 0x7f44ab0034de google::LogMessage::Flush() at ??:0
@ 0x7f44ab004a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f44adcc718d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f44adcbff0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f44b59687bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f44b594299c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f44b5943178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f44b5984df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f44b59a382b std::function<>::operator()() at ??:0
@ 0x7f44b5942b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f44b5943f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f44b5940579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f44b5984640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f44ad2553d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f44ad255c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f44ad2573f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f44ad246512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f44ae88a6db start_thread at ??:0
@ 0x7f44a9b0161f clone at ??:0
W20260811 22:25:05.782603 23197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:05.783058 23197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:05.783135 23197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:05.789458 23197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:05.789541 23197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:05.789636 23197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:25:05.798249 23197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:44389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=33159
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:05.799796 23197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:05.801242 23197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:05.813185 23286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:05.813284 23197 server_base.cc:1034] running on GCE node
W20260811 22:25:05.813201 23293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:05.813437 23287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:05.814028 23197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:05.814884 23197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:05.816104 23197 hybrid_clock.cc:648] HybridClock initialized: now 1786487105816080 us; error 30 us; skew 500 ppm
Aug 11 22:25:05 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487105, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:05.819630 23197 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:25:05.820855 23197 webserver.cc:460] Webserver started at http://127.26.125.1:33159/ using document root <none> and password file <none>
I20260811 22:25:05.821633 23197 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:05.821732 23197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:05.827010 23197 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:05.829442 23306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:05.831054 23197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:05.831185 23197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "ba8a171dfbb14cc38eb02d5506ae3456"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "cf0c991e6fabf2921d7af78f72506b07"
server_key_iv: "eee5f2e81296a1a23ea935cf52083dbe"
server_key_version: "encryptionkey@0"
I20260811 22:25:05.831779 23197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:05.848727 23197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:05.852286 23197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:05.852496 23197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:05.853132 23197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:05.854341 23316 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:25:05.860634 23197 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:25:05.860759 23197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260811 22:25:05.860841 23197 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:25:05.862891 23197 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:25:05.862973 23197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 22:25:05.863376 23316 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Bootstrap starting.
I20260811 22:25:05.873401 23197 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:44389
I20260811 22:25:05.873698 23518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:44389 every 8 connection(s)
I20260811 22:25:05.874591 23197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:05.879933 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 23197
I20260811 22:25:05.880690 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:40803
--local_ip_for_outbound_sockets=127.26.125.2
--tserver_master_addrs=127.26.125.62:44939
--webserver_port=34961
--webserver_interface=127.26.125.2
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:25:05.886442 23316 log.cc:824] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:05.895242 23316 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:25:05.895658 23316 tablet_bootstrap.cc:492] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Bootstrap complete.
I20260811 22:25:05.896616 23316 ts_tablet_manager.cc:1397] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Time spent bootstrapping tablet: real 0.034s user 0.027s sys 0.004s
I20260811 22:25:05.899242 23316 raft_consensus.cc:348] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:25:05.899705 23316 raft_consensus.cc:729] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8a171dfbb14cc38eb02d5506ae3456, State: Initialized, Role: FOLLOWER
I20260811 22:25:05.900226 23316 consensus_queue.cc:260] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:25:05.901237 23316 ts_tablet_manager.cc:1428] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 22:25:05.964329 23522 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:05.876751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:56063)
0811 22:25:05.877286 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:05.877344 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:05.878195 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:05.963431 (+ 85236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":139,"threads_started":1}
W20260811 22:25:05.965030 23520 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:44939 (0 consecutive failures): Network error: Failed to ping master at 127.26.125.62:44939: Client connection negotiation failed: client connection to 127.26.125.62:44939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:25:05.965852 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:05.965585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:58093)
0811 22:25:05.965715 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:05.965789 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 22:25:05.966519 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:05.966331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:59741)
0811 22:25:05.966424 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:05.966467 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:25:05.966681 23520 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:25:06.026930 23523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:06.027245 23523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:06.027295 23523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:06.031467 23523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:06.031549 23523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:06.031615 23523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:25:06.037366 23523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:40803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=34961
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:06.038607 23523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:06.039791 23523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:06.050531 23539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:06.050520 23535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:06.050590 23534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:06.050810 23523 server_base.cc:1034] running on GCE node
I20260811 22:25:06.051476 23523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:06.052276 23523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:06.053481 23523 hybrid_clock.cc:648] HybridClock initialized: now 1786487106053455 us; error 43 us; skew 500 ppm
Aug 11 22:25:06 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:06.056656 23523 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:25:06.057703 23523 webserver.cc:460] Webserver started at http://127.26.125.2:34961/ using document root <none> and password file <none>
I20260811 22:25:06.058305 23523 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:06.058389 23523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:06.062817 23523 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:06.065191 23554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:06.066596 23523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:25:06.066730 23523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "c8976bed2865d2be27c73254d2c4f8a8"
server_key_iv: "135b6a40f47d156d4398a789b39bdd27"
server_key_version: "encryptionkey@0"
I20260811 22:25:06.067380 23523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:06.111550 23523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:06.115473 23523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:06.115691 23523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:06.116360 23523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:06.117293 23523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:06.117344 23523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:06.117410 23523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:06.117437 23523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:06.126746 23523 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:40803
I20260811 22:25:06.126763 23776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:40803 every 8 connection(s)
I20260811 22:25:06.127933 23523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:06.132292 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:06.130399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:42671)
0811 22:25:06.131237 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:06.131450 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":123,"threads_started":1}
W20260811 22:25:06.132819 23778 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:44939 (0 consecutive failures): Network error: Failed to ping master at 127.26.125.62:44939: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
I20260811 22:25:06.133589 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:06.133271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:33017)
0811 22:25:06.133453 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:06.133514 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 22:25:06.134519 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:06.134159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:59301)
0811 22:25:06.134404 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:06.134460 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260811 22:25:06.134752 23778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:25:06.135010 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 23523
I20260811 22:25:06.135478 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:45463
--local_ip_for_outbound_sockets=127.26.125.3
--tserver_master_addrs=127.26.125.62:44939
--webserver_port=40395
--webserver_interface=127.26.125.3
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:06.256209 23784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:06.256601 23784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:06.256668 23784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:06.261081 23784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:06.261174 23784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:06.261349 23784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:25:06.267439 23784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:45463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=40395
--tserver_master_addrs=127.26.125.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:06.268654 23784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:06.269884 23784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:06.280949 23793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:06.280918 23792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:06.281057 23795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:06.281121 23784 server_base.cc:1034] running on GCE node
I20260811 22:25:06.281747 23784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:06.282495 23784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:06.283675 23784 hybrid_clock.cc:648] HybridClock initialized: now 1786487106283629 us; error 59 us; skew 500 ppm
Aug 11 22:25:06 dist-test-slave-5bn2 krb5kdc[21126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:06.286777 23784 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:25:06.287925 23784 webserver.cc:460] Webserver started at http://127.26.125.3:40395/ using document root <none> and password file <none>
I20260811 22:25:06.288582 23784 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:06.288659 23784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:06.293197 23784 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:25:06.295650 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:06.297122 23784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:25:06.297274 23784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "1977726fa0a94c7d85b35bc9d54ce61c"
format_stamp: "Formatted at 2026-08-11 22:24:23 on dist-test-slave-5bn2"
server_key: "d57245a3360de3eaafce5f80c42edb99"
server_key_iv: "6b980bd91dfddc0b9f3ee7214740d059"
server_key_version: "encryptionkey@0"
I20260811 22:25:06.297931 23784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:06.311679 23784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:06.315207 23784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:06.315457 23784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:06.316112 23784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:06.316998 23784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:06.317045 23784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:06.317090 23784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:06.317118 23784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:06.326108 23784 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:45463
I20260811 22:25:06.326160 24001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:45463 every 8 connection(s)
I20260811 22:25:06.327297 23784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:06.331449 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:06.329730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:60085)
0811 22:25:06.330524 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:06.330654 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":130,"threads_started":1}
W20260811 22:25:06.332012 24002 heartbeater.cc:644] Failed to heartbeat to 127.26.125.62:44939 (0 consecutive failures): Network error: Failed to ping master at 127.26.125.62:44939: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
I20260811 22:25:06.332613 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:06.332395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:40659)
0811 22:25:06.332504 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:06.332554 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 22:25:06.333319 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:06.333089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:53077)
0811 22:25:06.333217 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:06.333266 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:25:06.333493 24002 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:25:06.334182 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 23784
I20260811 22:25:06.967926 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:06.967436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:49277)
0811 22:25:06.967730 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:06.967802 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":109,"threads_started":1}
I20260811 22:25:07.135967 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:07.135441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:46199)
0811 22:25:07.135727 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:07.135820 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":77,"threads_started":1}
I20260811 22:25:07.217286 24012 raft_consensus.cc:482] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:07.217486 24012 raft_consensus.cc:504] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:25:07.218652 24012 leader_election.cc:288] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803), 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
Aug 11 22:25:07 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487105, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.2@KRBTEST.COM
Aug 11 22:25:07 dist-test-slave-5bn2 krb5kdc[21126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487105, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.3@KRBTEST.COM
I20260811 22:25:07.233119 24013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:07.219122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:33061 (local address 127.26.125.2:40803)
0811 22:25:07.219662 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:07.219689 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:07.219728 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:25:07.220421 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:07.220426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:07.220790 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:07.221067 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:07.222440 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:07.223497 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:07.224832 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:07.225419 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:07.225586 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:07.228604 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:07.228610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:07.228624 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:07.228655 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:07.230435 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:07.231057 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:07.231063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:07.231068 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:07.231189 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:07.231541 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:07.231544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:07.231545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:07.231983 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:07.232105 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:07.232671 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 22:25:07.232931 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":102,"threads_started":1}
I20260811 22:25:07.233105 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:07.218952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:40803 (local address 127.26.125.1:33061)
0811 22:25:07.219114 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:07.219129 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:07.219251 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:07.221167 (+ 1916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:07.221173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:07.221682 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:07.222239 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:07.222254 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:07.223657 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:07.223659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:07.224712 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:07.224724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:07.225549 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:07.225552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:07.225735 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:07.226465 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:07.226477 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:07.228354 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:07.230569 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:07.230576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:07.230592 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:07.230949 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:07.231280 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:07.231284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:07.231286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:07.231441 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:07.232097 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:07.232103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:07.232357 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 22:25:07.232521 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:25:07.232901 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":285,"spinlock_wait_cycles":30592}
I20260811 22:25:07.233147 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:07.219223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:45463 (local address 127.26.125.1:50593)
0811 22:25:07.219485 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:07.219498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:07.219595 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:07.221438 (+ 1843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:07.221440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:07.221679 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:07.222238 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:07.222254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:07.223654 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:07.223658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:07.224715 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:07.224727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:07.225507 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:07.225511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:07.225732 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:07.226428 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:07.226462 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:07.228677 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:07.230677 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:07.230680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:07.230683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:07.230987 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:07.231321 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:07.231324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:07.231326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:07.231432 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:07.232098 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:07.232105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:07.232357 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 22:25:07.232521 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:25:07.232976 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":300,"spinlock_wait_cycles":95232,"thread_start_us":67,"threads_started":1}
I20260811 22:25:07.233134 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:07.219434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:50593 (local address 127.26.125.3:45463)
0811 22:25:07.219766 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:07.219778 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:07.219811 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:25:07.220442 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:07.220449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:07.220793 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:07.221047 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:07.222424 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:07.223506 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:07.224839 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:07.225381 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:07.225534 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:07.228899 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:07.228904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:07.228919 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:07.228953 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:07.230554 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:07.231102 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:07.231107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:07.231112 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:07.231231 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:07.231566 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:07.231568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:07.231570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:07.231991 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:07.232105 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:07.232665 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 22:25:07.232944 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":135,"threads_started":1}
W20260811 22:25:07.235859 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
W20260811 22:25:07.236095 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
I20260811 22:25:07.236162 23313 leader_election.cc:302] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba8a171dfbb14cc38eb02d5506ae3456; no voters: 1977726fa0a94c7d85b35bc9d54ce61c, 83a9ea189ef84652b6bbdd41ad8ecfaa
I20260811 22:25:07.236428 24012 raft_consensus.cc:2726] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:25:07.334626 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:07.334157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:60591)
0811 22:25:07.334453 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:07.334519 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":98,"threads_started":1}
I20260811 22:25:07.969247 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:07.968802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:42389)
0811 22:25:07.969077 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:07.969146 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
I20260811 22:25:08.137369 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:08.136889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:53153)
0811 22:25:08.137195 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:08.137263 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":87,"threads_started":1}
I20260811 22:25:08.336045 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:08.335460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:37815)
0811 22:25:08.335816 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:08.335889 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":86,"threads_started":1}
I20260811 22:25:08.970731 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:08.970213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:50151)
0811 22:25:08.970501 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:08.970578 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
I20260811 22:25:08.999753 24031 raft_consensus.cc:482] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:08.999907 24031 raft_consensus.cc:504] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:25:09.000363 24031 leader_election.cc:288] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803), 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
W20260811 22:25:09.001350 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
W20260811 22:25:09.001492 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
I20260811 22:25:09.001578 23313 leader_election.cc:302] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba8a171dfbb14cc38eb02d5506ae3456; no voters: 1977726fa0a94c7d85b35bc9d54ce61c, 83a9ea189ef84652b6bbdd41ad8ecfaa
I20260811 22:25:09.001731 24031 raft_consensus.cc:2726] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:25:09.138788 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:09.138272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:36425)
0811 22:25:09.138598 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:09.138664 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":147,"threads_started":1}
I20260811 22:25:09.337329 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:09.336878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:49715)
0811 22:25:09.337163 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:09.337228 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
I20260811 22:25:09.972227 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:09.971700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:39673)
0811 22:25:09.972048 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:09.972120 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":84,"threads_started":1}
I20260811 22:25:10.140184 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:10.139717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:46493)
0811 22:25:10.140001 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:10.140079 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
I20260811 22:25:10.338581 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:10.338113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:40247)
0811 22:25:10.338377 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:10.338440 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260811 22:25:10.973721 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:10.973210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:60347)
0811 22:25:10.973546 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:10.973614 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":84,"threads_started":1}
I20260811 22:25:11.141389 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:11.140960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:48181)
0811 22:25:11.141237 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:11.141291 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":99,"threads_started":1}
I20260811 22:25:11.340060 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:11.339584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:51919)
0811 22:25:11.339853 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:11.339935 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":82,"threads_started":1}
I20260811 22:25:11.975049 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:11.974551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:34171)
0811 22:25:11.974836 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:11.974945 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
I20260811 22:25:12.142653 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:12.142178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:38329)
0811 22:25:12.142487 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:12.142551 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":80,"threads_started":1}
I20260811 22:25:12.157219 24052 raft_consensus.cc:482] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:12.157367 24052 raft_consensus.cc:504] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:25:12.157754 24052 leader_election.cc:288] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803), 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
W20260811 22:25:12.158636 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
W20260811 22:25:12.158797 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
I20260811 22:25:12.158867 23313 leader_election.cc:302] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba8a171dfbb14cc38eb02d5506ae3456; no voters: 1977726fa0a94c7d85b35bc9d54ce61c, 83a9ea189ef84652b6bbdd41ad8ecfaa
I20260811 22:25:12.159037 24052 raft_consensus.cc:2726] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:25:12.341260 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:12.340818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:50051)
0811 22:25:12.341097 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:12.341157 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
I20260811 22:25:12.976186 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:12.975770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:49701)
0811 22:25:12.976024 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:12.976084 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260811 22:25:13.144025 24057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:13.143511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:43305)
0811 22:25:13.143847 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:13.143913 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":77,"threads_started":1}
I20260811 22:25:13.342528 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:13.342013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:38959)
0811 22:25:13.342364 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:13.342425 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":88,"threads_started":1}
I20260811 22:25:13.977535 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:13.977069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:58743)
0811 22:25:13.977361 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:13.977427 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":95,"threads_started":1}
I20260811 22:25:14.145327 24061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:14.144866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:46177)
0811 22:25:14.145159 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:14.145225 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":85,"threads_started":1}
I20260811 22:25:14.343819 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:14.343366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:57745)
0811 22:25:14.343653 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:14.343713 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
I20260811 22:25:14.978749 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:14.978297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:51517)
0811 22:25:14.978589 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:14.978647 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":79,"threads_started":1}
I20260811 22:25:15.146567 24065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:15.146143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:34747)
0811 22:25:15.146389 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:15.146452 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":78,"threads_started":1}
I20260811 22:25:15.345180 24066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:15.344689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:40515)
0811 22:25:15.344999 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:15.345072 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
I20260811 22:25:15.980165 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:15.979750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:59965)
0811 22:25:15.980013 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:15.980066 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260811 22:25:16.095506 24069 raft_consensus.cc:482] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:16.095656 24069 raft_consensus.cc:504] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:25:16.096076 24069 leader_election.cc:288] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803), 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
W20260811 22:25:16.096982 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
W20260811 22:25:16.097115 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
I20260811 22:25:16.097165 23313 leader_election.cc:302] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba8a171dfbb14cc38eb02d5506ae3456; no voters: 1977726fa0a94c7d85b35bc9d54ce61c, 83a9ea189ef84652b6bbdd41ad8ecfaa
I20260811 22:25:16.097330 24069 raft_consensus.cc:2726] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:25:16.147886 24070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:16.147413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:54019)
0811 22:25:16.147722 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:16.147783 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":100,"threads_started":1}
I20260811 22:25:16.346396 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:16.345948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:36269)
0811 22:25:16.346234 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:16.346291 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
I20260811 22:25:16.981343 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:16.980904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:57291)
0811 22:25:16.981181 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:16.981240 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":100,"threads_started":1}
I20260811 22:25:17.149085 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:17.148648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:44871)
0811 22:25:17.148910 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:17.148981 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260811 22:25:17.347618 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:17.347207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:60091)
0811 22:25:17.347455 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:17.347514 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260811 22:25:17.962561 24077 raft_consensus.cc:482] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:17.962703 24077 raft_consensus.cc:504] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:25:17.963145 24077 leader_election.cc:288] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803), 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
W20260811 22:25:17.963996 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
W20260811 22:25:17.964131 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
I20260811 22:25:17.964181 23313 leader_election.cc:302] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba8a171dfbb14cc38eb02d5506ae3456; no voters: 1977726fa0a94c7d85b35bc9d54ce61c, 83a9ea189ef84652b6bbdd41ad8ecfaa
I20260811 22:25:17.964341 24077 raft_consensus.cc:2726] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:25:17.982793 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:17.982187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:46761)
0811 22:25:17.982494 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:17.982681 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":174,"threads_started":1}
I20260811 22:25:18.150437 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:18.149950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:56603)
0811 22:25:18.150269 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:18.150333 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":114,"threads_started":1}
I20260811 22:25:18.348819 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:18.348382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:52205)
0811 22:25:18.348652 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:18.348712 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
I20260811 22:25:18.984090 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:18.983620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:57121)
0811 22:25:18.983906 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:18.983976 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
I20260811 22:25:19.151772 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:19.151321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:40367)
0811 22:25:19.151603 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:19.151669 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
I20260811 22:25:19.349993 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:19.349523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:60155)
0811 22:25:19.349794 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:19.349863 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260811 22:25:19.811914 24090 raft_consensus.cc:482] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:19.812062 24090 raft_consensus.cc:504] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a9ea189ef84652b6bbdd41ad8ecfaa" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 40803 } } peers { permanent_uuid: "1977726fa0a94c7d85b35bc9d54ce61c" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 45463 } } peers { permanent_uuid: "ba8a171dfbb14cc38eb02d5506ae3456" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 44389 } }
I20260811 22:25:19.812498 24090 leader_election.cc:288] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803), 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463)
W20260811 22:25:19.813496 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83a9ea189ef84652b6bbdd41ad8ecfaa (127.26.125.2:40803): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
W20260811 22:25:19.813611 23313 leader_election.cc:341] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1977726fa0a94c7d85b35bc9d54ce61c (127.26.125.3:45463): Not found: Tablet not found: 75b73ed531ea44dc91dc887be4215382
I20260811 22:25:19.813645 23313 leader_election.cc:302] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba8a171dfbb14cc38eb02d5506ae3456; no voters: 1977726fa0a94c7d85b35bc9d54ce61c, 83a9ea189ef84652b6bbdd41ad8ecfaa
I20260811 22:25:19.813793 24090 raft_consensus.cc:2726] T 75b73ed531ea44dc91dc887be4215382 P ba8a171dfbb14cc38eb02d5506ae3456 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:25:19.985349 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:19.984880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:54415)
0811 22:25:19.985181 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:19.985244 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":104,"threads_started":1}
I20260811 22:25:20.153005 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:20.152529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:53037)
0811 22:25:20.152829 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:20.152899 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":108,"threads_started":1}
I20260811 22:25:20.351231 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:20.350786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:48793)
0811 22:25:20.351073 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:20.351131 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":119,"threads_started":1}
I20260811 22:25:20.986809 24096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:20.986289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.1:55985)
0811 22:25:20.986644 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:20.986706 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":159,"threads_started":1}
I20260811 22:25:21.154321 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:21.153894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.2:52865)
0811 22:25:21.154166 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:21.154221 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":104,"threads_started":1}
I20260811 22:25:21.341290 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 23197
I20260811 22:25:21.361464 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:21.360152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:44939 (local address 127.26.125.3:58045)
0811 22:25:21.361266 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:21.361343 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.62:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1018,"thread_start_us":866,"threads_started":1}
I20260811 22:25:21.369555 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 23523
I20260811 22:25:21.391006 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 23784
2026-08-11T22:25:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59799 ms)
[----------] 30 tests from SecurityITest (253107 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:21 dist-test-slave-5bn2 krb5kdc[24103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:21 dist-test-slave-5bn2 krb5kdc[24103](info): set up 2 sockets
Aug 11 22:25:21 dist-test-slave-5bn2 krb5kdc[24103](info): commencing operation
krb5kdc: starting...
W20260811 22:25:22.594458 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:22 dist-test-slave-5bn2 krb5kdc[24103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:25:22.731585 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:38363
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:38363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:22.851255 24128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:22.851562 24128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:22.851624 24128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:22.855808 24128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:22.855901 24128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:22.855930 24128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:22.855952 24128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:22.855983 24128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:22.861755 24128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:38363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:38363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:22.863080 24128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:22.864207 24128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:22.874157 24134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:22.874243 24128 server_base.cc:1034] running on GCE node
W20260811 22:25:22.874174 24137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:22.874379 24135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:22.875147 24128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:22.876384 24128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:22.877578 24128 hybrid_clock.cc:648] HybridClock initialized: now 1786487122877545 us; error 45 us; skew 500 ppm
Aug 11 22:25:22 dist-test-slave-5bn2 krb5kdc[24103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487122, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:22.880927 24128 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:22.881969 24128 webserver.cc:460] Webserver started at http://127.26.125.62:35105/ using document root <none> and password file <none>
I20260811 22:25:22.882618 24128 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:22.882692 24128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:22.882962 24128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:22.885192 24128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "d4281014d3d24076825c770c6111d089"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-5bn2"
server_key: "146aebb7a2e77696666efba92cb6e662"
server_key_iv: "ac7320bae4a48bd6a4f3470d67d43be8"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.885803 24128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "d4281014d3d24076825c770c6111d089"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-5bn2"
server_key: "146aebb7a2e77696666efba92cb6e662"
server_key_iv: "ac7320bae4a48bd6a4f3470d67d43be8"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.889921 24128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 22:25:22.892820 24151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:22.894284 24128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:22.894434 24128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "d4281014d3d24076825c770c6111d089"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-5bn2"
server_key: "146aebb7a2e77696666efba92cb6e662"
server_key_iv: "ac7320bae4a48bd6a4f3470d67d43be8"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.894569 24128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:22.904917 24128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:22.908576 24128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:22.908784 24128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:22.916249 24128 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:38363
I20260811 22:25:22.916266 24257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:38363 every 8 connection(s)
I20260811 22:25:22.917330 24128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:22.919827 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 24128
I20260811 22:25:22.919968 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:22.920290 27124 external_mini_cluster.cc:1428] Setting key 3e40c19d88cd5cbc4c44d183069ccc48
I20260811 22:25:22.920454 24259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:22.928452 24259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089: Bootstrap starting.
Aug 11 22:25:22 dist-test-slave-5bn2 krb5kdc[24103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:22.931360 24259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:22.932315 24259 log.cc:824] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:22.934813 24259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089: No bootstrap required, opened a new log
I20260811 22:25:22.936712 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:22.921793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56086 (local address 127.26.125.62:38363)
0811 22:25:22.922287 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:22.922307 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:22.922353 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:25:22.923460 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:22.923496 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:22.923968 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:22.924376 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:22.925129 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:22.926632 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:22.927636 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:22.928337 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:22.928516 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:22.931431 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:22.931439 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:22.931458 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:22.931508 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:22.933311 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:22.933801 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:22.933808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:22.933815 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:22.933968 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:22.934257 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:22.934260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:22.934262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:22.934818 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:22.935061 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:22.935344 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:25:22.935683 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":156,"threads_started":1}
I20260811 22:25:22.939414 24259 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4281014d3d24076825c770c6111d089" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38363 } }
I20260811 22:25:22.939669 24259 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:22.939714 24259 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4281014d3d24076825c770c6111d089, State: Initialized, Role: FOLLOWER
I20260811 22:25:22.940124 24259 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4281014d3d24076825c770c6111d089" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38363 } }
I20260811 22:25:22.940250 24259 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:22.940294 24259 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:22.940367 24259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:22.941476 24259 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4281014d3d24076825c770c6111d089" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38363 } }
I20260811 22:25:22.941792 24259 leader_election.cc:302] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4281014d3d24076825c770c6111d089; no voters:
I20260811 22:25:22.942108 24259 leader_election.cc:288] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:22.942224 24266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:22.942507 24266 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [term 1 LEADER]: Becoming Leader. State: Replica: d4281014d3d24076825c770c6111d089, State: Running, Role: LEADER
I20260811 22:25:22.942884 24266 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4281014d3d24076825c770c6111d089" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38363 } }
I20260811 22:25:22.943403 24259 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:22.944522 24269 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4281014d3d24076825c770c6111d089. Latest consensus state: current_term: 1 leader_uuid: "d4281014d3d24076825c770c6111d089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4281014d3d24076825c770c6111d089" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38363 } } }
I20260811 22:25:22.944639 24269 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:22.945009 24268 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4281014d3d24076825c770c6111d089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4281014d3d24076825c770c6111d089" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38363 } } }
I20260811 22:25:22.945073 24280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:22.945134 24268 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:22.948669 24280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:22.955013 24280 catalog_manager.cc:1347] Generated new cluster ID: 8c4ab6659089423883dc14b61e58000c
I20260811 22:25:22.955188 24280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:22.974769 24280 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:22.975893 24280 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:22.983476 24280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089: Generated new TSK 0
I20260811 22:25:22.984308 24280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:25:23.030036 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38363
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:23.160261 24306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:23.160682 24306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:23.160748 24306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:23.165009 24306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:23.165091 24306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:23.165150 24306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:25:23.170540 24306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:23.171662 24306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:23.172701 24306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:23.182713 24319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.182751 24312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.182751 24313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:23.182911 24306 server_base.cc:1034] running on GCE node
I20260811 22:25:23.183471 24306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:23.184234 24306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:23.185426 24306 hybrid_clock.cc:648] HybridClock initialized: now 1786487123185390 us; error 47 us; skew 500 ppm
Aug 11 22:25:23 dist-test-slave-5bn2 krb5kdc[24103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:23.188503 24306 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:25:23.189482 24306 webserver.cc:460] Webserver started at http://127.26.125.1:44199/ using document root <none> and password file <none>
I20260811 22:25:23.190101 24306 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:23.190167 24306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:23.190352 24306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:23.192587 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "737555c279784a8381df5926a1ba9082"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "191f5597a24c663f839d3ace65ef83ab"
server_key_iv: "c7941ee3069b9b3104d26220e9b135a0"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.193159 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "737555c279784a8381df5926a1ba9082"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "191f5597a24c663f839d3ace65ef83ab"
server_key_iv: "c7941ee3069b9b3104d26220e9b135a0"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.197355 24306 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:23.200207 24331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.201635 24306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:23.201757 24306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "737555c279784a8381df5926a1ba9082"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "191f5597a24c663f839d3ace65ef83ab"
server_key_iv: "c7941ee3069b9b3104d26220e9b135a0"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.201865 24306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:23.221872 24306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:23.225412 24306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:23.225592 24306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:23.226215 24306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:23.227133 24306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:23.227185 24306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.227226 24306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:23.227246 24306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.236208 24306 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:41107
I20260811 22:25:23.236238 24523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:41107 every 8 connection(s)
I20260811 22:25:23.237280 24306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:23.239503 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 24306
I20260811 22:25:23.239647 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:23.239933 27124 external_mini_cluster.cc:1428] Setting key 33357fbd88664c15a9b710e44fc5a981
Aug 11 22:25:23 dist-test-slave-5bn2 krb5kdc[24103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:25:23.253753 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.239396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:44817 (local address 127.26.125.62:38363)
0811 22:25:23.239562 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.239566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.240575 (+ 1009us) server_negotiation.cc:407] Connection header received
0811 22:25:23.241970 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.241974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.242029 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.242112 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:23.243619 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.244222 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.245906 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.246161 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.246256 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.249659 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:23.249663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:23.249665 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:23.249697 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:23.251303 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.251903 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.251906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.251907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.252001 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.252406 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.252409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.252411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.252646 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.252741 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.253373 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.253573 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:25:23.254701 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.239882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:38363 (local address 127.26.125.1:44817)
0811 22:25:23.240401 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:23.240464 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:23.241736 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:23.242291 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:23.242299 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:23.242693 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:23.243439 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.243462 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.244350 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.244354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.245752 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.245764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.246342 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.246346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.246592 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.247325 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:23.247348 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:23.249387 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:23.251448 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.251456 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.251479 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.251786 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.252095 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.252098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.252101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.252308 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.252757 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:23.252766 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:23.253079 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.253275 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.253676 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":136,"threads_started":1}
I20260811 22:25:23.256506 24525 heartbeater.cc:347] Connected to a master server at 127.26.125.62:38363
I20260811 22:25:23.256839 24525 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:23.257526 24525 heartbeater.cc:511] Master 127.26.125.62:38363 requested a full tablet report, sending...
I20260811 22:25:23.259662 24194 ts_manager.cc:194] Registered new tserver with Master: 737555c279784a8381df5926a1ba9082 (127.26.125.1:41107)
I20260811 22:25:23.261405 24194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:44817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:25:23.272382 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38363
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:23.395089 24534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:23.395407 24534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:23.395453 24534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:23.399896 24534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:23.399978 24534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:23.400050 24534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:25:23.405889 24534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:23.407145 24534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:23.408208 24534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:23.418444 24540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:23.418642 24534 server_base.cc:1034] running on GCE node
W20260811 22:25:23.418514 24541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.418449 24547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:23.419296 24534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:23.420017 24534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:23.421209 24534 hybrid_clock.cc:648] HybridClock initialized: now 1786487123421182 us; error 37 us; skew 500 ppm
Aug 11 22:25:23 dist-test-slave-5bn2 krb5kdc[24103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:23.424528 24534 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:25:23.425624 24534 webserver.cc:460] Webserver started at http://127.26.125.2:42737/ using document root <none> and password file <none>
I20260811 22:25:23.426254 24534 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:23.426330 24534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:23.426577 24534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:23.428867 24534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "6a66703d83bf4bba81f063727e72ebdd"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "08a8aab48fb4781554f5b821d38c83bd"
server_key_iv: "3c616a4a6458adffbfe81da0cdabf2d8"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.429484 24534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "6a66703d83bf4bba81f063727e72ebdd"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "08a8aab48fb4781554f5b821d38c83bd"
server_key_iv: "3c616a4a6458adffbfe81da0cdabf2d8"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.433737 24534 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:23.436605 24556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.438060 24534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:23.438261 24534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "6a66703d83bf4bba81f063727e72ebdd"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "08a8aab48fb4781554f5b821d38c83bd"
server_key_iv: "3c616a4a6458adffbfe81da0cdabf2d8"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.438426 24534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:23.453235 24534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:23.456836 24534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:23.457046 24534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:23.457692 24534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:23.458573 24534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:23.458622 24534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.458678 24534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:23.458703 24534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.467399 24534 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:42001
I20260811 22:25:23.467420 24766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:42001 every 8 connection(s)
I20260811 22:25:23.468382 24534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:23.471076 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 24534
I20260811 22:25:23.471215 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:23.471560 27124 external_mini_cluster.cc:1428] Setting key 2282809ea59e523f7edf920bf9a6a997
Aug 11 22:25:23 dist-test-slave-5bn2 krb5kdc[24103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:25:23.484261 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.470307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:47735 (local address 127.26.125.62:38363)
0811 22:25:23.470434 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.470438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.471484 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 22:25:23.472888 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.472891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.472938 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.473006 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:23.474281 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.475032 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.476766 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.477024 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.477130 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.480176 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:23.480180 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:23.480182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:23.480213 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:23.481758 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.482366 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.482371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.482372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.482459 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.482916 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.482920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.482922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.483159 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.483274 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.483990 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.484125 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:25:23.485180 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.470709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:38363 (local address 127.26.125.2:47735)
0811 22:25:23.471291 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:23.471354 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:23.472697 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:23.473218 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:23.473227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:23.473608 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:23.474119 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.474134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.475165 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.475169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.476601 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.476614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.477188 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.477191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.477357 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.477966 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:23.477986 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:23.479953 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:23.481908 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.481915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.481936 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.482255 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.482611 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.482614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.482615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.482803 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.483291 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:23.483299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:23.483706 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.483917 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.484240 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":180,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:25:23.486529 24767 heartbeater.cc:347] Connected to a master server at 127.26.125.62:38363
I20260811 22:25:23.486790 24767 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:23.487370 24767 heartbeater.cc:511] Master 127.26.125.62:38363 requested a full tablet report, sending...
I20260811 22:25:23.488556 24194 ts_manager.cc:194] Registered new tserver with Master: 6a66703d83bf4bba81f063727e72ebdd (127.26.125.2:42001)
I20260811 22:25:23.489171 24194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:47735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:25:23.504276 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38363
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:23.627378 24775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:23.627704 24775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:23.627770 24775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:23.632084 24775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:23.632174 24775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:23.632261 24775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:25:23.638024 24775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:23.639308 24775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:23.640506 24775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:23.651230 24786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.651504 24781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.651234 24783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:23.652005 24775 server_base.cc:1034] running on GCE node
I20260811 22:25:23.652509 24775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:23.653285 24775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:23.654590 24775 hybrid_clock.cc:648] HybridClock initialized: now 1786487123654575 us; error 37 us; skew 500 ppm
Aug 11 22:25:23 dist-test-slave-5bn2 krb5kdc[24103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:23.658393 24775 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:25:23.659575 24775 webserver.cc:460] Webserver started at http://127.26.125.3:39893/ using document root <none> and password file <none>
I20260811 22:25:23.660264 24775 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:23.660347 24775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:23.660600 24775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:23.663067 24775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "6b019b3cb6e24213a4f957f38f2c2c4d"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "19ba143db18bf062ed2384ed6159c437"
server_key_iv: "78cddbe6ccaba92c4e081834bd9583e1"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.663694 24775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "6b019b3cb6e24213a4f957f38f2c2c4d"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "19ba143db18bf062ed2384ed6159c437"
server_key_iv: "78cddbe6ccaba92c4e081834bd9583e1"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.668188 24775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:23.671272 24807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.672890 24775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:25:23.673044 24775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "6b019b3cb6e24213a4f957f38f2c2c4d"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-5bn2"
server_key: "19ba143db18bf062ed2384ed6159c437"
server_key_iv: "78cddbe6ccaba92c4e081834bd9583e1"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.673170 24775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:23.694216 24775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:23.697708 24775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:23.697933 24775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:23.698676 24775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:23.699923 24775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:23.699988 24775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.700053 24775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:23.700086 24775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.710764 24775 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:44529
I20260811 22:25:23.710790 24993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:44529 every 8 connection(s)
I20260811 22:25:23.711869 24775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:23.713006 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 24775
I20260811 22:25:23.713148 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:23.713474 27124 external_mini_cluster.cc:1428] Setting key 33903e179ba1da48c709aec74b73ee1d
Aug 11 22:25:23 dist-test-slave-5bn2 krb5kdc[24103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:23.726432 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.713977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:52793 (local address 127.26.125.62:38363)
0811 22:25:23.714138 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.714142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.715253 (+ 1111us) server_negotiation.cc:407] Connection header received
0811 22:25:23.716630 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.716632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.716691 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.716771 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:23.718023 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.718594 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.719867 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.720120 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.720209 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.722889 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:23.722894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:23.722897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:23.722927 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:23.724300 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.724844 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.724846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.724847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.724929 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.725255 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.725257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.725259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.725448 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.725529 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.726105 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.726244 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:25:23.727607 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.714536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:38363 (local address 127.26.125.3:52793)
0811 22:25:23.715091 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:23.715151 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:23.716393 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:23.716934 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:23.716941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:23.717344 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:23.717870 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.717885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.718726 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.718729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.719741 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.719752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.720259 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.720262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.720415 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.721047 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:23.721065 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:23.722673 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:23.724440 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.724446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.724463 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.724755 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.725009 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.725012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.725014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.725183 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.725534 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:23.725540 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:23.725821 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.726028 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.726393 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":137,"threads_started":1}
I20260811 22:25:23.728684 24994 heartbeater.cc:347] Connected to a master server at 127.26.125.62:38363
I20260811 22:25:23.728942 24994 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:23.729455 24994 heartbeater.cc:511] Master 127.26.125.62:38363 requested a full tablet report, sending...
I20260811 22:25:23.730599 24194 ts_manager.cc:194] Registered new tserver with Master: 6b019b3cb6e24213a4f957f38f2c2c4d (127.26.125.3:44529)
I20260811 22:25:23.731237 24194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:52793
I20260811 22:25:23.740023 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:23.750260 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.742440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.26.125.62:38363)
0811 22:25:23.742578 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.742582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.742745 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:25:23.742869 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.742873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.742931 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.743011 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:23.743873 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.744532 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.745430 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.745655 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.745746 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.747167 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:23.747170 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:23.747173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:23.747199 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:23.748624 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.749045 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.749049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.749050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.749178 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.749447 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.749450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.749451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.749708 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.749824 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.749999 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.750130 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:25:23.753391 24194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:23.756579 24194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:23.772341 25019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.766300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:44529 (local address 127.0.0.1:44326)
0811 22:25:23.767478 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:23.767492 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:23.767576 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:23.768493 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:23.768496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:23.768516 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:23.768615 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.768621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.769964 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.769968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.771030 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.771040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.771910 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.771913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.772007 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.772032 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.772099 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.772172 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"thread_start_us":82,"threads_started":1}
I20260811 22:25:23.772372 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.767200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.26.125.3:44529)
0811 22:25:23.768026 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.768035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.768055 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:25:23.768131 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.768136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.768333 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.768492 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:23.768724 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.769813 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.771150 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.771765 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.771906 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.772068 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.772127 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.772226 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":105,"threads_started":1}
I20260811 22:25:23.772495 25022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.766406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46758 (local address 127.26.125.2:42001)
0811 22:25:23.766962 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.766975 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.767003 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:25:23.767261 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.767267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.767532 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.767701 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:23.768220 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.769183 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.770604 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.771547 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.771656 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.771819 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.772256 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.772346 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":173,"threads_started":1}
I20260811 22:25:23.772976 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.766251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:42001 (local address 127.0.0.1:46758)
0811 22:25:23.766702 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:23.766765 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:23.766940 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:23.767908 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:23.767915 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:23.767941 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:23.768098 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.768105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.769300 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.769304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.770468 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.770479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.771682 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.771686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.771802 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.771857 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.771966 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.772109 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
I20260811 22:25:23.773365 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.767084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:41107 (local address 127.0.0.1:36176)
0811 22:25:23.767892 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:23.767905 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:23.768064 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:23.768731 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:23.768733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:23.768747 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:23.768834 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.768839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.770308 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.770311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.771756 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.771767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.772968 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.772972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.773073 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.773096 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.773168 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.773246 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":97,"threads_started":1}
I20260811 22:25:23.773713 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.767565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.26.125.1:41107)
0811 22:25:23.767830 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.767837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.767998 (+ 161us) server_negotiation.cc:407] Connection header received
0811 22:25:23.768202 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.768207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.768357 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.768491 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:23.768943 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.770040 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.771898 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.772700 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.772772 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.772897 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.773203 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.773308 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":64,"threads_started":1}
I20260811 22:25:23.775870 24891 tablet_service.cc:1467] Processing CreateTablet for tablet bceafdba85574294ade834e025f30dc6 (DEFAULT_TABLE table=test-table [id=6d38e104fd5d40d89da28731ce40b2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:23.776417 24419 tablet_service.cc:1467] Processing CreateTablet for tablet bceafdba85574294ade834e025f30dc6 (DEFAULT_TABLE table=test-table [id=6d38e104fd5d40d89da28731ce40b2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:23.776417 24643 tablet_service.cc:1467] Processing CreateTablet for tablet bceafdba85574294ade834e025f30dc6 (DEFAULT_TABLE table=test-table [id=6d38e104fd5d40d89da28731ce40b2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:23.776947 24891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bceafdba85574294ade834e025f30dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:23.777513 24643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bceafdba85574294ade834e025f30dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:23.777513 24419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bceafdba85574294ade834e025f30dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:23.783901 25036 tablet_bootstrap.cc:492] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Bootstrap starting.
I20260811 22:25:23.785177 25037 tablet_bootstrap.cc:492] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082: Bootstrap starting.
I20260811 22:25:23.787102 25036 tablet_bootstrap.cc:654] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:23.787259 25038 tablet_bootstrap.cc:492] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Bootstrap starting.
I20260811 22:25:23.787575 25037 tablet_bootstrap.cc:654] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:23.788548 25037 log.cc:824] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:23.789139 25036 log.cc:824] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:23.790033 25038 tablet_bootstrap.cc:654] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:23.790897 25038 log.cc:824] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:23.791106 25037 tablet_bootstrap.cc:492] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082: No bootstrap required, opened a new log
I20260811 22:25:23.791354 25037 ts_tablet_manager.cc:1397] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:25:23.791555 25036 tablet_bootstrap.cc:492] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: No bootstrap required, opened a new log
I20260811 22:25:23.791783 25036 ts_tablet_manager.cc:1397] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 22:25:23.792784 25038 tablet_bootstrap.cc:492] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: No bootstrap required, opened a new log
I20260811 22:25:23.792986 25038 ts_tablet_manager.cc:1397] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:23.795974 25037 raft_consensus.cc:348] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:23.796085 25038 raft_consensus.cc:348] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:23.796288 25037 raft_consensus.cc:374] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:23.796348 25038 raft_consensus.cc:374] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:23.796358 25037 raft_consensus.cc:729] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 737555c279784a8381df5926a1ba9082, State: Initialized, Role: FOLLOWER
I20260811 22:25:23.796396 25038 raft_consensus.cc:729] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a66703d83bf4bba81f063727e72ebdd, State: Initialized, Role: FOLLOWER
I20260811 22:25:23.795820 25036 raft_consensus.cc:348] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:23.796737 25036 raft_consensus.cc:374] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:23.796811 25036 raft_consensus.cc:729] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b019b3cb6e24213a4f957f38f2c2c4d, State: Initialized, Role: FOLLOWER
I20260811 22:25:23.796890 25037 consensus_queue.cc:260] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:23.796904 25038 consensus_queue.cc:260] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:23.797232 25036 consensus_queue.cc:260] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:23.798033 24525 heartbeater.cc:503] Master 127.26.125.62:38363 was elected leader, sending a full tablet report...
I20260811 22:25:23.798092 25036 ts_tablet_manager.cc:1428] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:25:23.798090 25038 ts_tablet_manager.cc:1428] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:25:23.798118 25037 ts_tablet_manager.cc:1428] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:25:23.798447 24994 heartbeater.cc:503] Master 127.26.125.62:38363 was elected leader, sending a full tablet report...
I20260811 22:25:23.798719 24767 heartbeater.cc:503] Master 127.26.125.62:38363 was elected leader, sending a full tablet report...
W20260811 22:25:23.963044 24995 tablet.cc:2369] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:25:23.969579 24770 tablet.cc:2369] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:25:23.988744 24528 tablet.cc:2369] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:24.023926 25043 raft_consensus.cc:482] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:24.024160 25043 raft_consensus.cc:504] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:24.025422 25043 leader_election.cc:288] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a66703d83bf4bba81f063727e72ebdd (127.26.125.2:42001), 6b019b3cb6e24213a4f957f38f2c2c4d (127.26.125.3:44529)
I20260811 22:25:24.029444 25022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.025786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:41001 (local address 127.26.125.2:42001)
0811 22:25:24.025943 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:24.025947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:24.026231 (+ 284us) server_negotiation.cc:407] Connection header received
0811 22:25:24.026402 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:24.026405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:24.026452 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:24.026521 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:24.026822 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.027341 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.028453 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.028869 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.028987 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.029045 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:24.029230 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 22:25:24.029301 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:25:24.029443 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.025848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:55341 (local address 127.26.125.3:44529)
0811 22:25:24.025979 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:24.025984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:24.026225 (+ 241us) server_negotiation.cc:407] Connection header received
0811 22:25:24.026404 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:24.026407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:24.026453 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:24.026517 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:24.026822 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.027354 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.028445 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.028879 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.028990 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.029045 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:24.029239 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 22:25:24.029309 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:25:24.029475 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.025735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:44529 (local address 127.26.125.1:55341)
0811 22:25:24.026100 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:24.026115 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:24.026280 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:24.026540 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:24.026542 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:24.026569 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:24.026722 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:24.026727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.027468 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.027472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:24.028316 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:24.028325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.028990 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.028994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:24.029134 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.029152 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:25:24.029214 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 22:25:24.029301 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":49,"spinlock_wait_cycles":43136,"thread_start_us":144,"threads_started":1}
I20260811 22:25:24.029488 25053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.025739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:42001 (local address 127.26.125.1:41001)
0811 22:25:24.026128 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:24.026139 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:24.026282 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:24.026544 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:24.026546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:24.026572 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:24.026719 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:24.026724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.027444 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.027448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:24.028328 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:24.028338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.028987 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.028991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:24.029092 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.029138 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:25:24.029203 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:25:24.029301 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":115,"spinlock_wait_cycles":13440,"thread_start_us":101,"threads_started":1}
I20260811 22:25:24.030267 24926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bceafdba85574294ade834e025f30dc6" candidate_uuid: "737555c279784a8381df5926a1ba9082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" is_pre_election: true
I20260811 22:25:24.030301 24679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bceafdba85574294ade834e025f30dc6" candidate_uuid: "737555c279784a8381df5926a1ba9082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a66703d83bf4bba81f063727e72ebdd" is_pre_election: true
I20260811 22:25:24.030575 24679 raft_consensus.cc:2454] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 737555c279784a8381df5926a1ba9082 in term 0.
I20260811 22:25:24.030579 24926 raft_consensus.cc:2454] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 737555c279784a8381df5926a1ba9082 in term 0.
I20260811 22:25:24.031061 24333 leader_election.cc:302] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a66703d83bf4bba81f063727e72ebdd, 737555c279784a8381df5926a1ba9082; no voters:
I20260811 22:25:24.031333 25043 raft_consensus.cc:2781] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:24.031432 25043 raft_consensus.cc:482] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:24.031484 25043 raft_consensus.cc:3037] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:24.032629 25043 raft_consensus.cc:504] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:24.032980 25043 leader_election.cc:288] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [CANDIDATE]: Term 1 election: Requested vote from peers 6a66703d83bf4bba81f063727e72ebdd (127.26.125.2:42001), 6b019b3cb6e24213a4f957f38f2c2c4d (127.26.125.3:44529)
I20260811 22:25:24.033427 24679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bceafdba85574294ade834e025f30dc6" candidate_uuid: "737555c279784a8381df5926a1ba9082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a66703d83bf4bba81f063727e72ebdd"
I20260811 22:25:24.033488 24926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bceafdba85574294ade834e025f30dc6" candidate_uuid: "737555c279784a8381df5926a1ba9082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d"
I20260811 22:25:24.033579 24679 raft_consensus.cc:3037] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:24.033603 24926 raft_consensus.cc:3037] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:24.034979 24926 raft_consensus.cc:2454] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 737555c279784a8381df5926a1ba9082 in term 1.
I20260811 22:25:24.034979 24679 raft_consensus.cc:2454] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 737555c279784a8381df5926a1ba9082 in term 1.
I20260811 22:25:24.035362 24333 leader_election.cc:302] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a66703d83bf4bba81f063727e72ebdd, 737555c279784a8381df5926a1ba9082; no voters:
I20260811 22:25:24.035579 25043 raft_consensus.cc:2781] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:24.035849 25043 raft_consensus.cc:686] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [term 1 LEADER]: Becoming Leader. State: Replica: 737555c279784a8381df5926a1ba9082, State: Running, Role: LEADER
I20260811 22:25:24.036230 25043 consensus_queue.cc:237] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } }
I20260811 22:25:24.039915 24189 catalog_manager.cc:5495] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 reported cstate change: term changed from 0 to 1, leader changed from <none> to 737555c279784a8381df5926a1ba9082 (127.26.125.1). New cstate: current_term: 1 leader_uuid: "737555c279784a8381df5926a1ba9082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "737555c279784a8381df5926a1ba9082" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 41107 } health_report { overall_health: HEALTHY } } }
I20260811 22:25:24.068948 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.065450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36188 (local address 127.26.125.1:41107)
0811 22:25:24.065586 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:24.065590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:24.065604 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:24.065704 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:24.065706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:24.065763 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:24.065843 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:24.066044 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.066542 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.067386 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.067610 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.067689 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.068000 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:24.068524 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:24.068641 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:24.068707 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 22:25:24.068773 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:25:24.075457 24679 raft_consensus.cc:1261] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 1 FOLLOWER]: Refusing update from remote peer 737555c279784a8381df5926a1ba9082: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:24.075457 24926 raft_consensus.cc:1261] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 1 FOLLOWER]: Refusing update from remote peer 737555c279784a8381df5926a1ba9082: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:24.076146 25043 consensus_queue.cc:1035] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a66703d83bf4bba81f063727e72ebdd" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 42001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:24.076371 25054 consensus_queue.cc:1035] T bceafdba85574294ade834e025f30dc6 P 737555c279784a8381df5926a1ba9082 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b019b3cb6e24213a4f957f38f2c2c4d" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:24.088122 25061 mvcc.cc:205] Tried to move back new op lower bound from 7317451260205121536 to 7317451260055113728. Current Snapshot: MvccSnapshot[applied={T|T < 7317451260205121536}]
I20260811 22:25:24.094228 24189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:24.094455 24189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:24.097115 24189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4281014d3d24076825c770c6111d089: Sending DeleteTablet for 3 replicas of tablet bceafdba85574294ade834e025f30dc6
I20260811 22:25:24.098104 24419 tablet_service.cc:1514] Processing DeleteTablet for tablet bceafdba85574294ade834e025f30dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:24 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:36176
I20260811 22:25:24.098091 24891 tablet_service.cc:1514] Processing DeleteTablet for tablet bceafdba85574294ade834e025f30dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:24 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:44326
I20260811 22:25:24.098099 24643 tablet_service.cc:1514] Processing DeleteTablet for tablet bceafdba85574294ade834e025f30dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:24 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:46758
I20260811 22:25:24.098728 25084 tablet_replica.cc:330] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: stopping tablet replica
I20260811 22:25:24.098928 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 24306
I20260811 22:25:24.099143 25084 raft_consensus.cc:2229] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:24.099499 25084 raft_consensus.cc:2258] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:24.099922 25073 tablet_replica.cc:330] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: stopping tablet replica
I20260811 22:25:24.100214 25073 raft_consensus.cc:2229] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:24.100492 25073 raft_consensus.cc:2258] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:24.103798 25084 ts_tablet_manager.cc:1905] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:24.104579 25073 ts_tablet_manager.cc:1905] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:24.106719 25084 ts_tablet_manager.cc:1918] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:24.106837 25084 log.cc:1196] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/bceafdba85574294ade834e025f30dc6
I20260811 22:25:24.107021 25073 ts_tablet_manager.cc:1918] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:24.107139 25073 log.cc:1196] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/bceafdba85574294ade834e025f30dc6
I20260811 22:25:24.107195 25084 ts_tablet_manager.cc:1939] T bceafdba85574294ade834e025f30dc6 P 6b019b3cb6e24213a4f957f38f2c2c4d: Deleting consensus metadata
I20260811 22:25:24.107494 25073 ts_tablet_manager.cc:1939] T bceafdba85574294ade834e025f30dc6 P 6a66703d83bf4bba81f063727e72ebdd: Deleting consensus metadata
I20260811 22:25:24.108697 24159 catalog_manager.cc:4841] TS 6b019b3cb6e24213a4f957f38f2c2c4d (127.26.125.3:44529): tablet bceafdba85574294ade834e025f30dc6 (table test-table [id=6d38e104fd5d40d89da28731ce40b2fa]) successfully deleted
I20260811 22:25:24.108795 24154 catalog_manager.cc:4841] TS 6a66703d83bf4bba81f063727e72ebdd (127.26.125.2:42001): tablet bceafdba85574294ade834e025f30dc6 (table test-table [id=6d38e104fd5d40d89da28731ce40b2fa]) successfully deleted
W20260811 22:25:24.110898 24154 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:41107 (error 108)
I20260811 22:25:24.110972 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 24534
I20260811 22:25:24.112206 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.111883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:41107 (local address 127.0.0.1:36190)
0811 22:25:24.112056 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:24.112117 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:41107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260811 22:25:24.112370 24154 catalog_manager.cc:4584] TS 737555c279784a8381df5926a1ba9082 (127.26.125.1:41107): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bceafdba85574294ade834e025f30dc6: Network error: Client connection negotiation failed: client connection to 127.26.125.1:41107: connect: Connection refused (error 111)
I20260811 22:25:24.120043 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 24775
I20260811 22:25:24.136246 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 24128
2026-08-11T22:25:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2740 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T22:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:24Z Disabled control of system clock
I20260811 22:25:24.175340 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:38835
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:38835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:25:24.295428 25091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:24.295743 25091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:24.295780 25091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:24.299888 25091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:24.299973 25091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:24.299986 25091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:24.300002 25091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:24.300016 25091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:24.305106 25091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:38835
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:38835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:24.306293 25091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:24.307852 25091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:24.316957 25101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:24.317044 25091 server_base.cc:1034] running on GCE node
W20260811 22:25:24.317013 25097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:24.316977 25096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:24.318003 25091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:24.319279 25091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:24.320451 25091 hybrid_clock.cc:648] HybridClock initialized: now 1786487124320423 us; error 41 us; skew 500 ppm
I20260811 22:25:24.322537 25091 webserver.cc:460] Webserver started at http://127.26.125.62:41477/ using document root <none> and password file <none>
I20260811 22:25:24.323201 25091 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:24.323279 25091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:24.323509 25091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:24.325714 25091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "ea3ec00ee9ab42269e55a496d171008a"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "cf858c995f90e19779a6734adcd48167"
server_key_iv: "b91524717189b2a91e141ba7dd2718a9"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.326293 25091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "ea3ec00ee9ab42269e55a496d171008a"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "cf858c995f90e19779a6734adcd48167"
server_key_iv: "b91524717189b2a91e141ba7dd2718a9"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.330189 25091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 22:25:24.332819 25112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:24.334170 25091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 22:25:24.334288 25091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "ea3ec00ee9ab42269e55a496d171008a"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "cf858c995f90e19779a6734adcd48167"
server_key_iv: "b91524717189b2a91e141ba7dd2718a9"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.334411 25091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:24.345357 25091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:24.346076 25091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:24.346262 25091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:24.353273 25091 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:38835
I20260811 22:25:24.353286 25198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:38835 every 8 connection(s)
I20260811 22:25:24.354269 25091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:24.357607 25201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:24.362306 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 25091
I20260811 22:25:24.362476 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:24.362752 27124 external_mini_cluster.cc:1428] Setting key e5afa6b375bacbbd538c5960f6feab4d
I20260811 22:25:24.364322 25201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a: Bootstrap starting.
I20260811 22:25:24.366989 25201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:24.367801 25201 log.cc:824] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:24.369784 25201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a: No bootstrap required, opened a new log
I20260811 22:25:24.372462 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.364337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44418 (local address 127.26.125.62:38835)
0811 22:25:24.365098 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:24.365117 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:24.365158 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:25:24.365847 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:24.365866 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:24.366319 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:24.366663 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:24.367213 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.368253 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.369151 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.369644 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.369797 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.370635 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:24.370641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:24.370656 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:24.371109 (+ 453us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:24.371170 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:24.371192 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:24.371280 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:24.371425 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 22:25:24.371704 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":162,"threads_started":1}
I20260811 22:25:24.373605 25201 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea3ec00ee9ab42269e55a496d171008a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38835 } }
I20260811 22:25:24.373848 25201 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:24.373908 25201 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea3ec00ee9ab42269e55a496d171008a, State: Initialized, Role: FOLLOWER
I20260811 22:25:24.374328 25201 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea3ec00ee9ab42269e55a496d171008a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38835 } }
I20260811 22:25:24.374468 25201 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:24.374531 25201 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:24.374632 25201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:24.375610 25201 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea3ec00ee9ab42269e55a496d171008a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38835 } }
I20260811 22:25:24.375962 25201 leader_election.cc:302] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea3ec00ee9ab42269e55a496d171008a; no voters:
I20260811 22:25:24.376266 25201 leader_election.cc:288] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:24.376381 25215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:24.376629 25215 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [term 1 LEADER]: Becoming Leader. State: Replica: ea3ec00ee9ab42269e55a496d171008a, State: Running, Role: LEADER
I20260811 22:25:24.376936 25215 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea3ec00ee9ab42269e55a496d171008a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38835 } }
I20260811 22:25:24.377617 25201 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:24.378836 25217 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea3ec00ee9ab42269e55a496d171008a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea3ec00ee9ab42269e55a496d171008a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38835 } } }
I20260811 22:25:24.378997 25217 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:24.379236 25218 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea3ec00ee9ab42269e55a496d171008a. Latest consensus state: current_term: 1 leader_uuid: "ea3ec00ee9ab42269e55a496d171008a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea3ec00ee9ab42269e55a496d171008a" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 38835 } } }
I20260811 22:25:24.379333 25218 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:24.379522 25237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:24.382555 25237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:24.389480 25237 catalog_manager.cc:1347] Generated new cluster ID: e233a9df6c1240189da6fa8288d9aecf
I20260811 22:25:24.389581 25237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:24.413688 25237 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:24.415067 25237 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:24.421720 25237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a: Generated new TSK 0
I20260811 22:25:24.422732 25237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:24.437872 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38835
--builtin_ntp_servers=127.26.125.20:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:25:24.574344 25253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:24.574676 25253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:24.574735 25253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:24.579043 25253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:24.579115 25253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:24.579193 25253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:25:24.584227 25253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:24.585466 25253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:24.587090 25253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:24.597822 25263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:24.597822 25267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:24.597822 25265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:24.598256 25253 server_base.cc:1034] running on GCE node
I20260811 22:25:24.598712 25253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:24.599493 25253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:24.600672 25253 hybrid_clock.cc:648] HybridClock initialized: now 1786487124600648 us; error 35 us; skew 500 ppm
I20260811 22:25:24.602816 25253 webserver.cc:460] Webserver started at http://127.26.125.1:39013/ using document root <none> and password file <none>
I20260811 22:25:24.603576 25253 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:24.603667 25253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:24.603919 25253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:24.606251 25253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "cd47ead7256f49f19490c0f288a69612"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "618f916b62331a618a3053e8861e5d69"
server_key_iv: "dfd1f004ced85c653491f5bff956a005"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.606920 25253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "cd47ead7256f49f19490c0f288a69612"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "618f916b62331a618a3053e8861e5d69"
server_key_iv: "dfd1f004ced85c653491f5bff956a005"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.611111 25253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:24.613845 25274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:24.615381 25253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:24.615518 25253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "cd47ead7256f49f19490c0f288a69612"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "618f916b62331a618a3053e8861e5d69"
server_key_iv: "dfd1f004ced85c653491f5bff956a005"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.615634 25253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:24.631620 25253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:24.632428 25253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:24.632627 25253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:24.633266 25253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:24.634371 25253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:24.634430 25253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:24.634492 25253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:24.634518 25253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:24.643426 25253 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:34703
I20260811 22:25:24.643479 25472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:34703 every 8 connection(s)
I20260811 22:25:24.644491 25253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:24.646760 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 25253
I20260811 22:25:24.646926 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:24.647233 27124 external_mini_cluster.cc:1428] Setting key 4ba5bb414819304ba01a79c2ac347743
I20260811 22:25:24.649281 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38835
--builtin_ntp_servers=127.26.125.20:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:25:24.655438 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.646662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:39123 (local address 127.26.125.62:38835)
0811 22:25:24.646833 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:24.646838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:24.648186 (+ 1348us) server_negotiation.cc:407] Connection header received
0811 22:25:24.649692 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:24.649695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:24.649750 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:24.649844 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:24.650769 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.651515 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.653199 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.653516 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.653634 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.654923 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:24.654927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:24.654930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:24.654947 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:24.654967 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:24.654975 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:24.655035 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:24.655195 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:25:24.655268 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:25:24.656415 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.647204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:38835 (local address 127.26.125.1:39123)
0811 22:25:24.648023 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:24.648082 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:24.649437 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:24.649988 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:24.649995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:24.650102 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:24.650627 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:24.650640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.651660 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.651664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:24.653052 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:24.653065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.653721 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.653726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:24.653938 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.654591 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:24.654593 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:24.654643 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:24.654727 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:24.655047 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:24.655057 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:24.655065 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:25:24.655185 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 22:25:24.655521 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":129,"threads_started":1}
I20260811 22:25:24.657655 25474 heartbeater.cc:347] Connected to a master server at 127.26.125.62:38835
I20260811 22:25:24.657913 25474 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:24.658489 25474 heartbeater.cc:511] Master 127.26.125.62:38835 requested a full tablet report, sending...
I20260811 22:25:24.660080 25140 ts_manager.cc:194] Registered new tserver with Master: cd47ead7256f49f19490c0f288a69612 (127.26.125.1:34703)
I20260811 22:25:24.661293 25140 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.1:39123
W20260811 22:25:24.776161 25479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:24.776468 25479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:24.776525 25479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:24.780782 25479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:24.780854 25479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:24.780936 25479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:25:24.785986 25479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:24.787278 25479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:24.788810 25479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:24.799156 25490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:24.799204 25493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:24.799263 25479 server_base.cc:1034] running on GCE node
W20260811 22:25:24.799211 25487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:24.799800 25479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:24.800486 25479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:24.801647 25479 hybrid_clock.cc:648] HybridClock initialized: now 1786487124801625 us; error 33 us; skew 500 ppm
I20260811 22:25:24.803639 25479 webserver.cc:460] Webserver started at http://127.26.125.2:32789/ using document root <none> and password file <none>
I20260811 22:25:24.804240 25479 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:24.804306 25479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:24.804536 25479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:24.806694 25479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "e0c4146a4d628fcfa83c3fb3fa950ff6"
server_key_iv: "1f3aab8a3e209dbf6ce697bcfc5ab885"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.807310 25479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "e0c4146a4d628fcfa83c3fb3fa950ff6"
server_key_iv: "1f3aab8a3e209dbf6ce697bcfc5ab885"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.811200 25479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:24.814035 25503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:24.815501 25479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:24.815636 25479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e"
format_stamp: "Formatted at 2026-08-11 22:25:24 on dist-test-slave-5bn2"
server_key: "e0c4146a4d628fcfa83c3fb3fa950ff6"
server_key_iv: "1f3aab8a3e209dbf6ce697bcfc5ab885"
server_key_version: "encryptionkey@0"
I20260811 22:25:24.815758 25479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:24.838351 25479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:24.839139 25479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:24.839331 25479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:24.839982 25479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:24.840932 25479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:24.840979 25479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:24.841019 25479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:24.841039 25479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:24.849404 25479 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:35527
I20260811 22:25:24.849431 25716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:35527 every 8 connection(s)
I20260811 22:25:24.850379 25479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:24.860289 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 25479
I20260811 22:25:24.860435 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:24.860661 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.852401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:41621 (local address 127.26.125.62:38835)
0811 22:25:24.852548 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:24.852553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:24.853822 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 22:25:24.855302 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:24.855305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:24.855354 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:24.855434 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:24.856407 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.856982 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.858435 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.858670 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.858768 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.860057 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:24.860061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:24.860063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:24.860080 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:24.860107 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:24.860116 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:24.860183 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:24.860433 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:25:24.860517 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:25:24.860785 27124 external_mini_cluster.cc:1428] Setting key caee3e406748a5e582161599d0bf25dc
I20260811 22:25:24.861804 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:24.852861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:38835 (local address 127.26.125.2:41621)
0811 22:25:24.853635 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:24.853696 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:24.855111 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:24.855639 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:24.855646 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:24.855737 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:24.856256 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:24.856269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.857151 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.857154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:24.858252 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:24.858264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:24.858825 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:24.858828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:24.859047 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:24.859744 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:24.859745 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:24.859797 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:24.859882 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:24.860202 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:24.860213 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:24.860220 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:25:24.860370 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 22:25:24.860771 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":119,"threads_started":1}
I20260811 22:25:24.863149 25717 heartbeater.cc:347] Connected to a master server at 127.26.125.62:38835
I20260811 22:25:24.863299 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38835
--builtin_ntp_servers=127.26.125.20:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:25:24.863401 25717 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:24.863989 25717 heartbeater.cc:511] Master 127.26.125.62:38835 requested a full tablet report, sending...
I20260811 22:25:24.865115 25140 ts_manager.cc:194] Registered new tserver with Master: 02043ce5d7c04bb58ccf7a523d8d9f5e (127.26.125.2:35527)
I20260811 22:25:24.865700 25140 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.2:41621
W20260811 22:25:24.982738 25721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:24.983096 25721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:24.983140 25721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:24.987310 25721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:24.987396 25721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:24.987453 25721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:25:24.992763 25721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:38835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:24.993973 25721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:24.995491 25721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:25.006168 25730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.006187 25728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.006187 25734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:25.006253 25721 server_base.cc:1034] running on GCE node
I20260811 22:25:25.007025 25721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:25.007750 25721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:25.008922 25721 hybrid_clock.cc:648] HybridClock initialized: now 1786487125008897 us; error 41 us; skew 500 ppm
I20260811 22:25:25.011001 25721 webserver.cc:460] Webserver started at http://127.26.125.3:38791/ using document root <none> and password file <none>
I20260811 22:25:25.011634 25721 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:25.011703 25721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:25.011943 25721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:25.014183 25721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "9edcfe16d6984acbb3f55693d49b9581"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-5bn2"
server_key: "4dcf3434c527c753bc2660b9184132e5"
server_key_iv: "2eb4e19a2a01f811f8b77cf8c5574276"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.014786 25721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "9edcfe16d6984acbb3f55693d49b9581"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-5bn2"
server_key: "4dcf3434c527c753bc2660b9184132e5"
server_key_iv: "2eb4e19a2a01f811f8b77cf8c5574276"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.018996 25721 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:25.021970 25750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.023382 25721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:25.023505 25721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "9edcfe16d6984acbb3f55693d49b9581"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-5bn2"
server_key: "4dcf3434c527c753bc2660b9184132e5"
server_key_iv: "2eb4e19a2a01f811f8b77cf8c5574276"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.023634 25721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:25.039004 25721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:25.039779 25721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:25.039968 25721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:25.040606 25721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:25.041651 25721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:25.041703 25721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.041746 25721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:25.041760 25721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.050448 25721 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:41915
I20260811 22:25:25.050486 25944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:41915 every 8 connection(s)
I20260811 22:25:25.051498 25721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:25.061697 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.053765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:46705 (local address 127.26.125.62:38835)
0811 22:25:25.053907 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.053911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.055217 (+ 1306us) server_negotiation.cc:407] Connection header received
0811 22:25:25.056707 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.056710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.056774 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.056852 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:25.057890 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.058428 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.059689 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.059916 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.060018 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.061169 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:25.061173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:25.061176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:25.061192 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:25.061214 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:25.061222 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:25.061282 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.061456 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.061533 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:25:25.061733 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 25721
I20260811 22:25:25.061841 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:25.062165 27124 external_mini_cluster.cc:1428] Setting key 67e51e1eef0ded79960c4a93326b18cf
I20260811 22:25:25.062662 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.054254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:38835 (local address 127.26.125.3:46705)
0811 22:25:25.055049 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.055099 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:25.056448 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:25.057067 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:25.057075 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:25.057181 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:25.057734 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.057747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.058571 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.058574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.059559 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.059570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.060077 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.060081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.060250 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.060880 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:25.060882 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:25.060930 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:25.061007 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:25.061290 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:25.061301 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:25.061308 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:25:25.061426 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:25:25.061765 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":121,"threads_started":1}
I20260811 22:25:25.064029 25947 heartbeater.cc:347] Connected to a master server at 127.26.125.62:38835
I20260811 22:25:25.064281 25947 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:25.064843 25947 heartbeater.cc:511] Master 127.26.125.62:38835 requested a full tablet report, sending...
I20260811 22:25:25.065898 25140 ts_manager.cc:194] Registered new tserver with Master: 9edcfe16d6984acbb3f55693d49b9581 (127.26.125.3:41915)
I20260811 22:25:25.066458 25140 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.3:46705
I20260811 22:25:25.076272 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:25.082124 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.078419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.26.125.62:38835)
0811 22:25:25.078565 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.078570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.078748 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:25:25.078894 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.078897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.078955 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.079024 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:25.079304 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.079859 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.080728 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.080958 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.081047 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.081741 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:25.081744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:25.081746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:25.081758 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:25.081776 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:25.081784 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:25.081835 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.081900 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.081959 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:25:25.085101 25140 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:25.088250 25140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:25.103591 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.097412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:41915 (local address 127.0.0.1:37942)
0811 22:25:25.098370 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.098427 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:25.098562 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:25.099181 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:25.099187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:25.099212 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:25.099361 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.099367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.100920 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.100923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.102150 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.102158 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.103002 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.103006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.103127 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.103167 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:25:25.103286 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:25:25.103423 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":22,"thread_start_us":102,"threads_started":1}
I20260811 22:25:25.103822 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.098180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54986 (local address 127.26.125.1:34703)
0811 22:25:25.098881 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.098892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.099172 (+ 280us) server_negotiation.cc:407] Connection header received
0811 22:25:25.099251 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.099257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.099454 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.099610 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:25.099822 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.100604 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.102235 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.103087 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.103190 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.103379 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.103544 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.103657 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":94,"threads_started":1}
I20260811 22:25:25.103920 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.097575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37942 (local address 127.26.125.3:41915)
0811 22:25:25.098630 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.098642 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.098668 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:25:25.098744 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.098749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.099003 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.099505 (+ 502us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:25.099642 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.100788 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.102257 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.102813 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.102988 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.103212 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.103670 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.103777 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":181,"threads_started":1}
I20260811 22:25:25.104174 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.097822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:34703 (local address 127.0.0.1:54986)
0811 22:25:25.098966 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.099015 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:25.099081 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:25.099596 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:25.099599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:25.099617 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:25.099714 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.099721 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.100719 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.100722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.102092 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.102104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.103293 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.103297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.103397 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.103418 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:25:25.103498 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:25:25.103564 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260811 22:25:25.104203 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.098302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.26.125.2:35527)
0811 22:25:25.098783 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.098795 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.099168 (+ 373us) server_negotiation.cc:407] Connection header received
0811 22:25:25.099251 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.099257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.099454 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.099610 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:25.100010 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.101077 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.102253 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.103087 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.103190 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.103379 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.103952 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.104063 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":173,"threads_started":1}
I20260811 22:25:25.103816 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.097465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:35527 (local address 127.0.0.1:35674)
0811 22:25:25.098949 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.098962 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:25.099063 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:25.099769 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:25.099772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:25.099790 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:25.099881 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.099887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.101198 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.101201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.102120 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.102130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.103443 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.103446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.103545 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.103570 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:25:25.103636 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:25:25.103698 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1421,"mutex_wait_us":24,"thread_start_us":88,"threads_started":1}
I20260811 22:25:25.107275 25590 tablet_service.cc:1467] Processing CreateTablet for tablet 9561019508034e7794ffc07b46ec6e32 (DEFAULT_TABLE table=test-table [id=a09041a1cf7748b4a06ea915eab25831]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:25.107384 25354 tablet_service.cc:1467] Processing CreateTablet for tablet 9561019508034e7794ffc07b46ec6e32 (DEFAULT_TABLE table=test-table [id=a09041a1cf7748b4a06ea915eab25831]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:25.107659 25833 tablet_service.cc:1467] Processing CreateTablet for tablet 9561019508034e7794ffc07b46ec6e32 (DEFAULT_TABLE table=test-table [id=a09041a1cf7748b4a06ea915eab25831]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:25.108327 25590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9561019508034e7794ffc07b46ec6e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:25.108471 25354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9561019508034e7794ffc07b46ec6e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:25.108497 25833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9561019508034e7794ffc07b46ec6e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:25.115500 25975 tablet_bootstrap.cc:492] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: Bootstrap starting.
I20260811 22:25:25.116549 25976 tablet_bootstrap.cc:492] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612: Bootstrap starting.
I20260811 22:25:25.116971 25977 tablet_bootstrap.cc:492] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: Bootstrap starting.
I20260811 22:25:25.118189 25975 tablet_bootstrap.cc:654] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:25.118901 25977 tablet_bootstrap.cc:654] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:25.119206 25975 log.cc:824] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:25.119930 25977 log.cc:824] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:25.120777 25976 tablet_bootstrap.cc:654] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:25.121642 25975 tablet_bootstrap.cc:492] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: No bootstrap required, opened a new log
I20260811 22:25:25.121666 25976 log.cc:824] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:25.121869 25975 ts_tablet_manager.cc:1397] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:25:25.121960 25977 tablet_bootstrap.cc:492] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: No bootstrap required, opened a new log
I20260811 22:25:25.122177 25977 ts_tablet_manager.cc:1397] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:25:25.123723 25976 tablet_bootstrap.cc:492] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612: No bootstrap required, opened a new log
I20260811 22:25:25.123914 25976 ts_tablet_manager.cc:1397] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 22:25:25.125514 25977 raft_consensus.cc:348] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.125742 25977 raft_consensus.cc:374] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:25.125814 25977 raft_consensus.cc:729] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edcfe16d6984acbb3f55693d49b9581, State: Initialized, Role: FOLLOWER
I20260811 22:25:25.126282 25977 consensus_queue.cc:260] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.126168 25975 raft_consensus.cc:348] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.126420 25975 raft_consensus.cc:374] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:25.126507 25975 raft_consensus.cc:729] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02043ce5d7c04bb58ccf7a523d8d9f5e, State: Initialized, Role: FOLLOWER
I20260811 22:25:25.127064 25975 consensus_queue.cc:260] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.127650 25977 ts_tablet_manager.cc:1428] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:25:25.128153 25947 heartbeater.cc:503] Master 127.26.125.62:38835 was elected leader, sending a full tablet report...
I20260811 22:25:25.128232 25975 ts_tablet_manager.cc:1428] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:25:25.128082 25976 raft_consensus.cc:348] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.128413 25976 raft_consensus.cc:374] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:25.128516 25976 raft_consensus.cc:729] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd47ead7256f49f19490c0f288a69612, State: Initialized, Role: FOLLOWER
I20260811 22:25:25.128620 25717 heartbeater.cc:503] Master 127.26.125.62:38835 was elected leader, sending a full tablet report...
I20260811 22:25:25.129009 25976 consensus_queue.cc:260] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.129669 25474 heartbeater.cc:503] Master 127.26.125.62:38835 was elected leader, sending a full tablet report...
I20260811 22:25:25.130877 25976 ts_tablet_manager.cc:1428] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 22:25:25.145777 25475 tablet.cc:2369] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:25.207990 25982 raft_consensus.cc:482] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:25.208240 25982 raft_consensus.cc:504] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.209511 25982 leader_election.cc:288] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02043ce5d7c04bb58ccf7a523d8d9f5e (127.26.125.2:35527), cd47ead7256f49f19490c0f288a69612 (127.26.125.1:34703)
I20260811 22:25:25.213351 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.209886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:51117 (local address 127.26.125.2:35527)
0811 22:25:25.210053 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.210057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.210074 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:25:25.210153 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.210156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.210201 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.210264 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:25.210590 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.211234 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.212376 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.212812 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.212905 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.212948 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.213147 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.213217 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:25:25.213333 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.209775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:35527 (local address 127.26.125.3:51117)
0811 22:25:25.209914 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.209931 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:25.210049 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:25.210287 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:25.210290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:25.210308 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:25.210439 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.210447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.211391 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.211395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.212253 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.212263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.212926 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.212930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.213028 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.213056 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:25:25.213122 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:25:25.213196 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 22:25:25.213476 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.209895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:40507 (local address 127.26.125.1:34703)
0811 22:25:25.210050 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.210055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.210426 (+ 371us) server_negotiation.cc:407] Connection header received
0811 22:25:25.210524 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.210528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.210577 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.210642 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:25.210941 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.211531 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.212539 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.212963 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.213075 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.213119 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.213289 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.213365 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:25:25.213639 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.209775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:34703 (local address 127.26.125.3:40507)
0811 22:25:25.210324 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.210336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:25.210444 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:25.210669 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:25.210671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:25.210689 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:25.210810 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.210816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.211645 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.211648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.212423 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.212433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.213074 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.213077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.213169 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.213192 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:25:25.213263 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:25:25.213323 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":159,"thread_start_us":169,"threads_started":1}
I20260811 22:25:25.214133 25630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9561019508034e7794ffc07b46ec6e32" candidate_uuid: "9edcfe16d6984acbb3f55693d49b9581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" is_pre_election: true
I20260811 22:25:25.214280 25392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9561019508034e7794ffc07b46ec6e32" candidate_uuid: "9edcfe16d6984acbb3f55693d49b9581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd47ead7256f49f19490c0f288a69612" is_pre_election: true
I20260811 22:25:25.214515 25630 raft_consensus.cc:2454] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9edcfe16d6984acbb3f55693d49b9581 in term 0.
I20260811 22:25:25.214514 25392 raft_consensus.cc:2454] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9edcfe16d6984acbb3f55693d49b9581 in term 0.
I20260811 22:25:25.215108 25758 leader_election.cc:302] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02043ce5d7c04bb58ccf7a523d8d9f5e, 9edcfe16d6984acbb3f55693d49b9581; no voters:
I20260811 22:25:25.215405 25982 raft_consensus.cc:2781] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:25.215502 25982 raft_consensus.cc:482] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:25.215534 25982 raft_consensus.cc:3037] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:25.216634 25982 raft_consensus.cc:504] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.217057 25982 leader_election.cc:288] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [CANDIDATE]: Term 1 election: Requested vote from peers 02043ce5d7c04bb58ccf7a523d8d9f5e (127.26.125.2:35527), cd47ead7256f49f19490c0f288a69612 (127.26.125.1:34703)
I20260811 22:25:25.217425 25630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9561019508034e7794ffc07b46ec6e32" candidate_uuid: "9edcfe16d6984acbb3f55693d49b9581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e"
I20260811 22:25:25.217464 25392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9561019508034e7794ffc07b46ec6e32" candidate_uuid: "9edcfe16d6984acbb3f55693d49b9581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd47ead7256f49f19490c0f288a69612"
I20260811 22:25:25.217552 25630 raft_consensus.cc:3037] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:25.217674 25392 raft_consensus.cc:3037] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:25.218667 25630 raft_consensus.cc:2454] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9edcfe16d6984acbb3f55693d49b9581 in term 1.
I20260811 22:25:25.218742 25392 raft_consensus.cc:2454] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9edcfe16d6984acbb3f55693d49b9581 in term 1.
I20260811 22:25:25.219112 25752 leader_election.cc:302] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9edcfe16d6984acbb3f55693d49b9581, cd47ead7256f49f19490c0f288a69612; no voters:
I20260811 22:25:25.219331 25982 raft_consensus.cc:2781] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:25.219595 25982 raft_consensus.cc:686] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 1 LEADER]: Becoming Leader. State: Replica: 9edcfe16d6984acbb3f55693d49b9581, State: Running, Role: LEADER
I20260811 22:25:25.220002 25982 consensus_queue.cc:237] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } }
I20260811 22:25:25.223492 25139 catalog_manager.cc:5495] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9edcfe16d6984acbb3f55693d49b9581 (127.26.125.3). New cstate: current_term: 1 leader_uuid: "9edcfe16d6984acbb3f55693d49b9581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edcfe16d6984acbb3f55693d49b9581" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 41915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 } health_report { overall_health: UNKNOWN } } }
I20260811 22:25:25.258424 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.254929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37954 (local address 127.26.125.3:41915)
0811 22:25:25.255080 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.255084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.255101 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:25:25.255205 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.255208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.255265 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.255360 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:25.255604 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.256099 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.256950 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.257163 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.257259 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.257579 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:25.258026 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:25.258138 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.258206 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.258257 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:25.264578 25630 raft_consensus.cc:1261] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 1 FOLLOWER]: Refusing update from remote peer 9edcfe16d6984acbb3f55693d49b9581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:25.264556 25392 raft_consensus.cc:1261] T 9561019508034e7794ffc07b46ec6e32 P cd47ead7256f49f19490c0f288a69612 [term 1 FOLLOWER]: Refusing update from remote peer 9edcfe16d6984acbb3f55693d49b9581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:25.265316 25991 consensus_queue.cc:1035] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd47ead7256f49f19490c0f288a69612" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 34703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:25.265502 25982 consensus_queue.cc:1035] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02043ce5d7c04bb58ccf7a523d8d9f5e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 35527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:25.275745 26000 mvcc.cc:205] Tried to move back new op lower bound from 7317451265076137984 to 7317451264904204288. Current Snapshot: MvccSnapshot[applied={T|T < 7317451265076137984}]
I20260811 22:25:25.276652 25997 mvcc.cc:205] Tried to move back new op lower bound from 7317451265076137984 to 7317451264904204288. Current Snapshot: MvccSnapshot[applied={T|T < 7317451265076137984 or (T in {7317451265076137984})}]
I20260811 22:25:25.282366 25138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:25.282579 25138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:25.285342 25138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea3ec00ee9ab42269e55a496d171008a: Sending DeleteTablet for 3 replicas of tablet 9561019508034e7794ffc07b46ec6e32
I20260811 22:25:25.286230 25833 tablet_service.cc:1514] Processing DeleteTablet for tablet 9561019508034e7794ffc07b46ec6e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:25 UTC) from {username='slave'} at 127.0.0.1:37942
I20260811 22:25:25.286242 25354 tablet_service.cc:1514] Processing DeleteTablet for tablet 9561019508034e7794ffc07b46ec6e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:25 UTC) from {username='slave'} at 127.0.0.1:54986
I20260811 22:25:25.286242 25590 tablet_service.cc:1514] Processing DeleteTablet for tablet 9561019508034e7794ffc07b46ec6e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:25 UTC) from {username='slave'} at 127.0.0.1:35674
I20260811 22:25:25.286887 26014 tablet_replica.cc:330] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: stopping tablet replica
I20260811 22:25:25.287269 26014 raft_consensus.cc:2229] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:25.287647 26014 raft_consensus.cc:2258] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:25.287839 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 25253
I20260811 22:25:25.304771 26016 tablet_replica.cc:330] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: stopping tablet replica
I20260811 22:25:25.305213 26016 raft_consensus.cc:2229] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:25.305658 26016 raft_consensus.cc:2258] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:25.305904 26014 ts_tablet_manager.cc:1905] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:25.307718 26016 ts_tablet_manager.cc:1905] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:25.309337 26014 ts_tablet_manager.cc:1918] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:25.309546 26014 log.cc:1196] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/9561019508034e7794ffc07b46ec6e32
I20260811 22:25:25.309913 26014 ts_tablet_manager.cc:1939] T 9561019508034e7794ffc07b46ec6e32 P 02043ce5d7c04bb58ccf7a523d8d9f5e: Deleting consensus metadata
I20260811 22:25:25.310942 26016 ts_tablet_manager.cc:1918] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:25.311089 26016 log.cc:1196] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/9561019508034e7794ffc07b46ec6e32
I20260811 22:25:25.311623 26016 ts_tablet_manager.cc:1939] T 9561019508034e7794ffc07b46ec6e32 P 9edcfe16d6984acbb3f55693d49b9581: Deleting consensus metadata
I20260811 22:25:25.311578 25117 catalog_manager.cc:4841] TS 02043ce5d7c04bb58ccf7a523d8d9f5e (127.26.125.2:35527): tablet 9561019508034e7794ffc07b46ec6e32 (table test-table [id=a09041a1cf7748b4a06ea915eab25831]) successfully deleted
I20260811 22:25:25.312875 25114 catalog_manager.cc:4841] TS 9edcfe16d6984acbb3f55693d49b9581 (127.26.125.3:41915): tablet 9561019508034e7794ffc07b46ec6e32 (table test-table [id=a09041a1cf7748b4a06ea915eab25831]) successfully deleted
W20260811 22:25:25.312992 25113 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:34703 (error 108)
I20260811 22:25:25.313879 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 25479
I20260811 22:25:25.314121 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.313780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:34703 (local address 127.0.0.1:55000)
0811 22:25:25.313932 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.314015 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:25:25.320991 25113 catalog_manager.cc:4584] TS cd47ead7256f49f19490c0f288a69612 (127.26.125.1:34703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9561019508034e7794ffc07b46ec6e32: Network error: Client connection negotiation failed: client connection to 127.26.125.1:34703: connect: Connection refused (error 111)
I20260811 22:25:25.341244 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 25721
I20260811 22:25:25.368886 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 25091
2026-08-11T22:25:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1235 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:25 dist-test-slave-5bn2 krb5kdc[26025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:25 dist-test-slave-5bn2 krb5kdc[26025](info): set up 2 sockets
Aug 11 22:25:25 dist-test-slave-5bn2 krb5kdc[26025](info): commencing operation
krb5kdc: starting...
W20260811 22:25:26.570659 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:26 dist-test-slave-5bn2 krb5kdc[26025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:25:26.709487 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:37139
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:37139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:26.828935 26043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:26.829236 26043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:26.829276 26043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:26.833422 26043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:26.833508 26043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:26.833526 26043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:26.833542 26043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:26.833556 26043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:26.839748 26043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:37139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:37139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:26.840945 26043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:26.842068 26043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:26.851136 26063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:26.851249 26043 server_base.cc:1034] running on GCE node
W20260811 22:25:26.851114 26059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:26.851121 26058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:26.852061 26043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:26.853328 26043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:26.854540 26043 hybrid_clock.cc:648] HybridClock initialized: now 1786487126854518 us; error 32 us; skew 500 ppm
Aug 11 22:25:26 dist-test-slave-5bn2 krb5kdc[26025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487126, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:26.869798 26043 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:26.870946 26043 webserver.cc:460] Webserver started at http://127.26.125.62:43169/ using document root <none> and password file <none>
I20260811 22:25:26.871553 26043 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:26.871621 26043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:26.871799 26043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:26.873951 26043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "ecd9600e3cb84149803b6e8b21ac08d0"
format_stamp: "Formatted at 2026-08-11 22:25:26 on dist-test-slave-5bn2"
server_key: "d6ce7276a98c91df53b4fe18aaae0638"
server_key_iv: "82232cb122abe302be9159a9af71be1d"
server_key_version: "encryptionkey@0"
I20260811 22:25:26.874518 26043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "ecd9600e3cb84149803b6e8b21ac08d0"
format_stamp: "Formatted at 2026-08-11 22:25:26 on dist-test-slave-5bn2"
server_key: "d6ce7276a98c91df53b4fe18aaae0638"
server_key_iv: "82232cb122abe302be9159a9af71be1d"
server_key_version: "encryptionkey@0"
I20260811 22:25:26.878602 26043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:26.881408 26071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:26.882841 26043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:26.883054 26043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "ecd9600e3cb84149803b6e8b21ac08d0"
format_stamp: "Formatted at 2026-08-11 22:25:26 on dist-test-slave-5bn2"
server_key: "d6ce7276a98c91df53b4fe18aaae0638"
server_key_iv: "82232cb122abe302be9159a9af71be1d"
server_key_version: "encryptionkey@0"
I20260811 22:25:26.883184 26043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:26.899569 26043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:26.903548 26043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:26.903757 26043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:26.911286 26043 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:37139
I20260811 22:25:26.911314 26160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:37139 every 8 connection(s)
I20260811 22:25:26.912362 26043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:26.915246 26162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:26.917964 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 26043
I20260811 22:25:26.918103 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:26.918354 27124 external_mini_cluster.cc:1428] Setting key fce4585c83a6bbf5799ed43280842c12
I20260811 22:25:26.921454 26162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0: Bootstrap starting.
I20260811 22:25:26.924134 26162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:26.925084 26162 log.cc:824] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:26.927541 26162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0: No bootstrap required, opened a new log
Aug 11 22:25:26 dist-test-slave-5bn2 krb5kdc[26025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:26.931155 26162 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd9600e3cb84149803b6e8b21ac08d0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37139 } }
I20260811 22:25:26.931398 26162 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:26.931455 26162 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd9600e3cb84149803b6e8b21ac08d0, State: Initialized, Role: FOLLOWER
I20260811 22:25:26.931989 26162 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd9600e3cb84149803b6e8b21ac08d0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37139 } }
I20260811 22:25:26.932159 26162 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:26.932222 26162 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:26.932318 26162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:26.933087 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.919902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.26.125.62:37139)
0811 22:25:26.920351 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.920366 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.920405 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:25:26.921054 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.921073 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.921424 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.921739 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:26.922705 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.923804 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.925008 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.925586 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.925756 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.928494 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:26.928499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:26.928514 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:26.928544 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:26.930161 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.930586 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.930591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.930597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.930715 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.931031 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.931034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.931035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.931499 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:26.931683 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.931978 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.932276 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":128,"threads_started":1}
I20260811 22:25:26.933259 26162 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd9600e3cb84149803b6e8b21ac08d0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37139 } }
I20260811 22:25:26.933599 26162 leader_election.cc:302] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecd9600e3cb84149803b6e8b21ac08d0; no voters:
I20260811 22:25:26.933893 26162 leader_election.cc:288] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:26.934019 26173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:26.934237 26173 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [term 1 LEADER]: Becoming Leader. State: Replica: ecd9600e3cb84149803b6e8b21ac08d0, State: Running, Role: LEADER
I20260811 22:25:26.934501 26173 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd9600e3cb84149803b6e8b21ac08d0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37139 } }
I20260811 22:25:26.935251 26162 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:26.935614 26174 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecd9600e3cb84149803b6e8b21ac08d0. Latest consensus state: current_term: 1 leader_uuid: "ecd9600e3cb84149803b6e8b21ac08d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd9600e3cb84149803b6e8b21ac08d0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37139 } } }
I20260811 22:25:26.935814 26174 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:26.935889 26176 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecd9600e3cb84149803b6e8b21ac08d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd9600e3cb84149803b6e8b21ac08d0" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37139 } } }
I20260811 22:25:26.935997 26176 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:26.936411 26181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:26.939754 26181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:26.946578 26181 catalog_manager.cc:1347] Generated new cluster ID: 7fa8f339fa1b46b3a3c73a54b316fa41
I20260811 22:25:26.946677 26181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:26.956243 26181 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:26.957304 26181 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:26.964797 26181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0: Generated new TSK 0
I20260811 22:25:26.965549 26181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:25:27.000208 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37139
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:27.122577 26206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:27.122926 26206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:27.122973 26206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:27.127130 26206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:27.127231 26206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:27.127317 26206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:25:27.133018 26206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:27.134227 26206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:27.135393 26206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:27.146790 26221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:27.146832 26218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:27.146790 26215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:27.147138 26206 server_base.cc:1034] running on GCE node
I20260811 22:25:27.147673 26206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:27.148387 26206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:27.149577 26206 hybrid_clock.cc:648] HybridClock initialized: now 1786487127149543 us; error 51 us; skew 500 ppm
Aug 11 22:25:27 dist-test-slave-5bn2 krb5kdc[26025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:27.152925 26206 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:25:27.153973 26206 webserver.cc:460] Webserver started at http://127.26.125.1:40175/ using document root <none> and password file <none>
I20260811 22:25:27.154594 26206 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:27.154667 26206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:27.154935 26206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:27.157265 26206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "d861633065ed44d18a446645a57604de"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "275fba8e9bb5d00aae1d3538e299e779"
server_key_iv: "743ecc61dbaae18c6f4b936ec54879b7"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.157876 26206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "d861633065ed44d18a446645a57604de"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "275fba8e9bb5d00aae1d3538e299e779"
server_key_iv: "743ecc61dbaae18c6f4b936ec54879b7"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.162076 26206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:27.164927 26237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.166348 26206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:27.166498 26206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "d861633065ed44d18a446645a57604de"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "275fba8e9bb5d00aae1d3538e299e779"
server_key_iv: "743ecc61dbaae18c6f4b936ec54879b7"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.166615 26206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:27.185083 26206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:27.188788 26206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:27.188997 26206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:27.189658 26206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:27.190595 26206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:27.190649 26206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.190706 26206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:27.190732 26206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.199921 26206 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:42785
I20260811 22:25:27.199942 26443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:42785 every 8 connection(s)
I20260811 22:25:27.201002 26206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:27.208760 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 26206
I20260811 22:25:27.208905 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:27.209236 27124 external_mini_cluster.cc:1428] Setting key 0d7590a4b19ffa2084371f12c8b3cd53
Aug 11 22:25:27 dist-test-slave-5bn2 krb5kdc[26025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:27.215428 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.202949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:38489 (local address 127.26.125.62:37139)
0811 22:25:27.203125 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.203129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.204064 (+ 935us) server_negotiation.cc:407] Connection header received
0811 22:25:27.205413 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.205417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.205467 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.205581 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:27.206794 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.207369 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.208583 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.208917 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.209018 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.211498 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:27.211502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:27.211504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:27.211533 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:27.213138 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.213746 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.213748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.213749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.213830 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.214202 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.214204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.214206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.214404 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:27.214477 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.215169 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.215294 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:27.216686 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.203370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:37139 (local address 127.26.125.1:38489)
0811 22:25:27.203899 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.203946 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:27.205208 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:27.205754 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:27.205761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:27.206134 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:27.206670 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.206685 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.207493 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.207496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.208466 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.208477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.209067 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.209071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.209222 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.209822 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:27.209839 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:27.211322 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:27.213256 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:27.213264 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:27.213284 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:27.213653 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:27.213912 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:27.213915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:27.213916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:27.214112 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:27.214503 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:27.214510 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:27.214884 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 22:25:27.215116 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 22:25:27.215490 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":133,"threads_started":1}
I20260811 22:25:27.218016 26447 heartbeater.cc:347] Connected to a master server at 127.26.125.62:37139
I20260811 22:25:27.218276 26447 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:27.218814 26447 heartbeater.cc:511] Master 127.26.125.62:37139 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:25:27.220428 26106 ts_manager.cc:194] Registered new tserver with Master: d861633065ed44d18a446645a57604de (127.26.125.1:42785)
I20260811 22:25:27.221671 26106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:38489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:25:27.241860 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37139
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:27.361990 26457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:27.362306 26457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:27.362372 26457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:27.366513 26457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:27.366601 26457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:27.366696 26457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:25:27.372381 26457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:27.373585 26457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:27.374722 26457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:27.384672 26464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:27.384804 26469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:27.384964 26457 server_base.cc:1034] running on GCE node
W20260811 22:25:27.384701 26465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:27.385555 26457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:27.386245 26457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:27.387483 26457 hybrid_clock.cc:648] HybridClock initialized: now 1786487127387452 us; error 39 us; skew 500 ppm
Aug 11 22:25:27 dist-test-slave-5bn2 krb5kdc[26025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:27.390573 26457 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:25:27.391628 26457 webserver.cc:460] Webserver started at http://127.26.125.2:42571/ using document root <none> and password file <none>
I20260811 22:25:27.392231 26457 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:27.392298 26457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:27.392511 26457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:27.394690 26457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "a55f1a3a4fab4722acf53768520e944c"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "132bb85024e8a57228bfefcca9157582"
server_key_iv: "50f52eb77d74dbd6025e47193ff1975d"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.395354 26457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "a55f1a3a4fab4722acf53768520e944c"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "132bb85024e8a57228bfefcca9157582"
server_key_iv: "50f52eb77d74dbd6025e47193ff1975d"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.399457 26457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:27.402245 26481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.403764 26457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:27.403889 26457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "a55f1a3a4fab4722acf53768520e944c"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "132bb85024e8a57228bfefcca9157582"
server_key_iv: "50f52eb77d74dbd6025e47193ff1975d"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.404006 26457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:27.415673 26457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:27.418895 26457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:27.419098 26457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:27.419708 26457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:27.420603 26457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:27.420653 26457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.420708 26457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:27.420733 26457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.429590 26457 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:45223
I20260811 22:25:27.429611 26678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:45223 every 8 connection(s)
I20260811 22:25:27.430621 26457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:27.440104 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 26457
I20260811 22:25:27.440258 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
Aug 11 22:25:27 dist-test-slave-5bn2 krb5kdc[26025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:27.440603 27124 external_mini_cluster.cc:1428] Setting key 3901927a0ec28f580295c5e6833f5fa8
I20260811 22:25:27.444809 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.432695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:56347 (local address 127.26.125.62:37139)
0811 22:25:27.432841 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.432845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.433672 (+ 827us) server_negotiation.cc:407] Connection header received
0811 22:25:27.435029 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.435032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.435084 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.435160 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:27.436504 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.437032 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.438249 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.438467 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.438546 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.441171 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:27.441175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:27.441177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:27.441204 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:27.442637 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.443196 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.443199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.443200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.443284 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.443644 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.443646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.443648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.443845 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:27.443939 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.444492 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.444630 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:25:27.445808 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.433051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:37139 (local address 127.26.125.2:56347)
0811 22:25:27.433515 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.433563 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:27.434772 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:27.435350 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:27.435359 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:27.435770 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:27.436350 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.436370 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.437149 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.437152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.438142 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.438152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.438587 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.438590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.438731 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.439306 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:27.439322 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:27.440997 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:27.442748 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:27.442755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:27.442774 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:27.443100 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:27.443371 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:27.443374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:27.443376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:27.443540 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:27.443947 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:27.443954 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:27.444239 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 22:25:27.444417 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 22:25:27.444803 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":126,"threads_started":1}
I20260811 22:25:27.447189 26679 heartbeater.cc:347] Connected to a master server at 127.26.125.62:37139
I20260811 22:25:27.447459 26679 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:27.448127 26679 heartbeater.cc:511] Master 127.26.125.62:37139 requested a full tablet report, sending...
I20260811 22:25:27.449326 26106 ts_manager.cc:194] Registered new tserver with Master: a55f1a3a4fab4722acf53768520e944c (127.26.125.2:45223)
I20260811 22:25:27.449944 26106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:56347
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:25:27.473454 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37139
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:40791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:27.595186 26684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:27.595508 26684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:27.595552 26684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:27.599725 26684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:27.599805 26684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:27.599871 26684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:25:27.605697 26684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:40791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:27.606935 26684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:27.608132 26684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:27.618688 26694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:27.618698 26696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:27.618754 26684 server_base.cc:1034] running on GCE node
W20260811 22:25:27.618662 26700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:27.619379 26684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:27.620103 26684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:27.621279 26684 hybrid_clock.cc:648] HybridClock initialized: now 1786487127621251 us; error 36 us; skew 500 ppm
Aug 11 22:25:27 dist-test-slave-5bn2 krb5kdc[26025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:27.624279 26684 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:25:27.625293 26684 webserver.cc:460] Webserver started at http://127.26.125.3:34769/ using document root <none> and password file <none>
I20260811 22:25:27.625901 26684 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:27.625973 26684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:27.626188 26684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:27.628453 26684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "1279e5892bf249468ca7d3a75489e6cd"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "43ef1cd8ff9f0e2424d00f25fd00ac38"
server_key_iv: "ef1e11c2e51ac77ba74fc9572bb802ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.629063 26684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "1279e5892bf249468ca7d3a75489e6cd"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "43ef1cd8ff9f0e2424d00f25fd00ac38"
server_key_iv: "ef1e11c2e51ac77ba74fc9572bb802ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.633540 26684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:27.636683 26710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.638110 26684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:27.638248 26684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "1279e5892bf249468ca7d3a75489e6cd"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-5bn2"
server_key: "43ef1cd8ff9f0e2424d00f25fd00ac38"
server_key_iv: "ef1e11c2e51ac77ba74fc9572bb802ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.638370 26684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:27.652709 26684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:27.656180 26684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:27.656376 26684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:27.656999 26684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:27.657915 26684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:27.657967 26684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.658025 26684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:27.658051 26684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.667847 26684 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:34801
I20260811 22:25:27.667873 26928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:34801 every 8 connection(s)
I20260811 22:25:27.668839 26684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:27.671736 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 26684
I20260811 22:25:27.671875 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:27.672221 27124 external_mini_cluster.cc:1428] Setting key 69c536f2d5b5240e0efa250fd72a8612
Aug 11 22:25:27 dist-test-slave-5bn2 krb5kdc[26025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:27.683256 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.670911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:54415 (local address 127.26.125.62:37139)
0811 22:25:27.671050 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.671054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.672002 (+ 948us) server_negotiation.cc:407] Connection header received
0811 22:25:27.673579 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.673582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.673648 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.673747 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:27.675239 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.675798 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.676911 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.677121 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.677193 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.679768 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:27.679771 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:27.679773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:27.679799 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:27.681222 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.681742 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.681744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.681746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.681833 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.682168 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.682170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.682171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.682360 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:27.682431 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.683005 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.683122 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:25:27.684186 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.671356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:37139 (local address 127.26.125.3:54415)
0811 22:25:27.671838 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.671897 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:27.673350 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:27.673941 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:27.673951 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:27.674487 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:27.675101 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.675118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.675908 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.675911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.676817 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.676827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.677260 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.677263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.677410 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.677982 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:27.677999 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:27.679589 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:27.681341 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:27.681347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:27.681364 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:27.681658 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:27.681918 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:27.681921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:27.681922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:27.682094 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:27.682432 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:27.682437 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:27.682750 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 22:25:27.682955 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 22:25:27.683274 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260811 22:25:27.685395 26929 heartbeater.cc:347] Connected to a master server at 127.26.125.62:37139
I20260811 22:25:27.685639 26929 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:27.686236 26929 heartbeater.cc:511] Master 127.26.125.62:37139 requested a full tablet report, sending...
I20260811 22:25:27.687371 26106 ts_manager.cc:194] Registered new tserver with Master: 1279e5892bf249468ca7d3a75489e6cd (127.26.125.3:34801)
I20260811 22:25:27.687970 26106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:54415
I20260811 22:25:27.698688 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:27.709062 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.700862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53560 (local address 127.26.125.62:37139)
0811 22:25:27.700984 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.700988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.701222 (+ 234us) server_negotiation.cc:407] Connection header received
0811 22:25:27.701317 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.701319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.701366 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.701436 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:27.702255 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.702814 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.703713 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.703965 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.704058 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.705757 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:27.705760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:27.705762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:27.705786 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:27.707159 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.707752 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.707755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.707756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.707834 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.708136 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.708139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.708140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.708383 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:27.708526 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.708805 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.708897 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:25:27.712606 26106 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:27.715807 26106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:27.731127 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.725403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.26.125.1:42785)
0811 22:25:27.725678 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.725685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.726048 (+ 363us) server_negotiation.cc:407] Connection header received
0811 22:25:27.726352 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.726358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.726582 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.726709 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:27.727243 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.728164 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.729488 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.730325 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.730385 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.730509 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.730878 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.730982 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260811 22:25:27.731118 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.725263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:42785 (local address 127.0.0.1:56016)
0811 22:25:27.725902 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.725956 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:27.726166 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:27.726938 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:27.726945 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:27.726970 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:27.727119 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.727126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.728268 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.728271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.729347 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.729358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.730585 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.730587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.730671 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.730700 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:25:27.730792 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:25:27.730948 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":1,"thread_start_us":101,"threads_started":1}
I20260811 22:25:27.731485 26949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.725398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:34801 (local address 127.0.0.1:56858)
0811 22:25:27.726355 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.726367 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:27.726460 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:27.727324 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:27.727327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:27.727346 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:27.727442 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.727448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.729016 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.729020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.729985 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.729995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.731098 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.731103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.731198 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.731220 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:25:27.731271 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 22:25:27.731341 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":175,"threads_started":1}
I20260811 22:25:27.732041 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.726053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:45223 (local address 127.0.0.1:57426)
0811 22:25:27.726802 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.726814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:27.726950 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:27.727525 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:27.727528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:27.727547 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:27.727626 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.727631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.728922 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.728926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.730136 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.730147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.731664 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.731667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.731750 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.731773 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:25:27.731842 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:25:27.731908 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260811 22:25:27.732116 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.725905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56858 (local address 127.26.125.3:34801)
0811 22:25:27.726938 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.726949 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.726973 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:25:27.727057 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.727061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.727198 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.727350 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:27.727583 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.728725 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.730129 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.730960 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.731681 (+ 721us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.731825 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.731879 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.731972 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":901,"thread_start_us":180,"threads_started":1}
I20260811 22:25:27.732530 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.726406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.26.125.2:45223)
0811 22:25:27.726711 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.726722 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.726921 (+ 199us) server_negotiation.cc:407] Connection header received
0811 22:25:27.727080 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.727086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.727286 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.727432 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:27.727740 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.728817 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.730264 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.731059 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.731148 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.731436 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.731884 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.731981 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":133,"threads_started":1}
I20260811 22:25:27.734755 26804 tablet_service.cc:1467] Processing CreateTablet for tablet f66c8b14069348fba0f8559ec48e0023 (DEFAULT_TABLE table=test-table [id=ee3442d47d634edb864e7e42e30d2e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:27.734680 26331 tablet_service.cc:1467] Processing CreateTablet for tablet f66c8b14069348fba0f8559ec48e0023 (DEFAULT_TABLE table=test-table [id=ee3442d47d634edb864e7e42e30d2e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:27.735842 26331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66c8b14069348fba0f8559ec48e0023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:27.735847 26804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66c8b14069348fba0f8559ec48e0023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:27.735680 26557 tablet_service.cc:1467] Processing CreateTablet for tablet f66c8b14069348fba0f8559ec48e0023 (DEFAULT_TABLE table=test-table [id=ee3442d47d634edb864e7e42e30d2e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:27.736675 26557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66c8b14069348fba0f8559ec48e0023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:27.742467 26956 tablet_bootstrap.cc:492] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: Bootstrap starting.
I20260811 22:25:27.743810 26958 tablet_bootstrap.cc:492] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de: Bootstrap starting.
I20260811 22:25:27.745589 26956 tablet_bootstrap.cc:654] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:27.746052 26959 tablet_bootstrap.cc:492] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: Bootstrap starting.
I20260811 22:25:27.746116 26958 tablet_bootstrap.cc:654] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:27.746997 26958 log.cc:824] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:27.747007 26956 log.cc:824] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:27.748654 26959 tablet_bootstrap.cc:654] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:27.749365 26956 tablet_bootstrap.cc:492] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: No bootstrap required, opened a new log
I20260811 22:25:27.749418 26958 tablet_bootstrap.cc:492] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de: No bootstrap required, opened a new log
I20260811 22:25:27.749569 26956 ts_tablet_manager.cc:1397] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:25:27.749542 26959 log.cc:824] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:27.749601 26958 ts_tablet_manager.cc:1397] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:27.751780 26959 tablet_bootstrap.cc:492] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: No bootstrap required, opened a new log
I20260811 22:25:27.751989 26959 ts_tablet_manager.cc:1397] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 22:25:27.752743 26956 raft_consensus.cc:348] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.753011 26956 raft_consensus.cc:374] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:27.753065 26956 raft_consensus.cc:729] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1279e5892bf249468ca7d3a75489e6cd, State: Initialized, Role: FOLLOWER
I20260811 22:25:27.753494 26956 consensus_queue.cc:260] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.753950 26958 raft_consensus.cc:348] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.754138 26929 heartbeater.cc:503] Master 127.26.125.62:37139 was elected leader, sending a full tablet report...
I20260811 22:25:27.754215 26958 raft_consensus.cc:374] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:27.754261 26958 raft_consensus.cc:729] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d861633065ed44d18a446645a57604de, State: Initialized, Role: FOLLOWER
I20260811 22:25:27.754612 26956 ts_tablet_manager.cc:1428] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:25:27.754684 26958 consensus_queue.cc:260] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.755702 26958 ts_tablet_manager.cc:1428] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:25:27.755973 26959 raft_consensus.cc:348] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.756199 26959 raft_consensus.cc:374] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:27.756232 26959 raft_consensus.cc:729] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55f1a3a4fab4722acf53768520e944c, State: Initialized, Role: FOLLOWER
I20260811 22:25:27.756546 26959 consensus_queue.cc:260] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.757161 26679 heartbeater.cc:503] Master 127.26.125.62:37139 was elected leader, sending a full tablet report...
I20260811 22:25:27.757304 26959 ts_tablet_manager.cc:1428] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:25:27.757372 26447 heartbeater.cc:503] Master 127.26.125.62:37139 was elected leader, sending a full tablet report...
I20260811 22:25:27.786826 26966 raft_consensus.cc:482] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:27.787196 26966 raft_consensus.cc:504] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.788618 26966 leader_election.cc:288] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d861633065ed44d18a446645a57604de (127.26.125.1:42785), a55f1a3a4fab4722acf53768520e944c (127.26.125.2:45223)
I20260811 22:25:27.793424 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.789257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:58531 (local address 127.26.125.2:45223)
0811 22:25:27.789462 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.789466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.789565 (+ 99us) server_negotiation.cc:407] Connection header received
0811 22:25:27.789683 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.789686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.789736 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.789808 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:27.790018 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.790705 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.792095 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.792729 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.792858 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.792915 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.793211 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.793285 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:25:27.793424 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.788991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:51979 (local address 127.26.125.1:42785)
0811 22:25:27.789117 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.789121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.789137 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:27.789195 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.789198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.789241 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.789327 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:27.789684 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.790464 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.792058 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.792729 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.792858 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.792915 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.793209 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.793285 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:25:27.793443 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.788831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:42785 (local address 127.26.125.3:51979)
0811 22:25:27.788959 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.788981 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:27.789074 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:27.789417 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:27.789421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:27.789442 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:27.789579 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.789586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.790592 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.790595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.791875 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.791889 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.792851 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.792855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.793061 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.793116 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 22:25:27.793185 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:25:27.793271 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":76}
I20260811 22:25:27.793454 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.789097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:45223 (local address 127.26.125.3:58531)
0811 22:25:27.789486 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.789499 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:27.789589 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:27.789822 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:27.789824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:27.789838 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:27.789928 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.789933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.790806 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.790809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.791961 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:27.791970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.792851 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.792855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:27.793035 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.793107 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 22:25:27.793180 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:25:27.793271 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":48,"thread_start_us":137,"threads_started":1}
I20260811 22:25:27.794248 26361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66c8b14069348fba0f8559ec48e0023" candidate_uuid: "1279e5892bf249468ca7d3a75489e6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d861633065ed44d18a446645a57604de" is_pre_election: true
I20260811 22:25:27.794250 26590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66c8b14069348fba0f8559ec48e0023" candidate_uuid: "1279e5892bf249468ca7d3a75489e6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55f1a3a4fab4722acf53768520e944c" is_pre_election: true
I20260811 22:25:27.794618 26361 raft_consensus.cc:2454] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1279e5892bf249468ca7d3a75489e6cd in term 0.
I20260811 22:25:27.794618 26590 raft_consensus.cc:2454] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1279e5892bf249468ca7d3a75489e6cd in term 0.
I20260811 22:25:27.795169 26717 leader_election.cc:302] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1279e5892bf249468ca7d3a75489e6cd, a55f1a3a4fab4722acf53768520e944c; no voters:
I20260811 22:25:27.795444 26966 raft_consensus.cc:2781] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:27.795549 26966 raft_consensus.cc:482] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:27.795584 26966 raft_consensus.cc:3037] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:27.796636 26966 raft_consensus.cc:504] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.796979 26966 leader_election.cc:288] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [CANDIDATE]: Term 1 election: Requested vote from peers d861633065ed44d18a446645a57604de (127.26.125.1:42785), a55f1a3a4fab4722acf53768520e944c (127.26.125.2:45223)
I20260811 22:25:27.797377 26361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66c8b14069348fba0f8559ec48e0023" candidate_uuid: "1279e5892bf249468ca7d3a75489e6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d861633065ed44d18a446645a57604de"
I20260811 22:25:27.797406 26590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66c8b14069348fba0f8559ec48e0023" candidate_uuid: "1279e5892bf249468ca7d3a75489e6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55f1a3a4fab4722acf53768520e944c"
I20260811 22:25:27.797498 26361 raft_consensus.cc:3037] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:27.797521 26590 raft_consensus.cc:3037] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:27.798544 26361 raft_consensus.cc:2454] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1279e5892bf249468ca7d3a75489e6cd in term 1.
I20260811 22:25:27.798594 26590 raft_consensus.cc:2454] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1279e5892bf249468ca7d3a75489e6cd in term 1.
I20260811 22:25:27.798911 26717 leader_election.cc:302] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1279e5892bf249468ca7d3a75489e6cd, d861633065ed44d18a446645a57604de; no voters:
I20260811 22:25:27.799095 26966 raft_consensus.cc:2781] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:27.799352 26966 raft_consensus.cc:686] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 1 LEADER]: Becoming Leader. State: Replica: 1279e5892bf249468ca7d3a75489e6cd, State: Running, Role: LEADER
I20260811 22:25:27.799770 26966 consensus_queue.cc:237] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } }
I20260811 22:25:27.803134 26106 catalog_manager.cc:5495] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1279e5892bf249468ca7d3a75489e6cd (127.26.125.3). New cstate: current_term: 1 leader_uuid: "1279e5892bf249468ca7d3a75489e6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1279e5892bf249468ca7d3a75489e6cd" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 34801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 } health_report { overall_health: UNKNOWN } } }
I20260811 22:25:27.832618 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.828864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56872 (local address 127.26.125.3:34801)
0811 22:25:27.829014 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.829018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.829034 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:27.829192 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.829194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.829253 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.829349 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:27.829631 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.830150 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.831065 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.831282 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.831375 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.831716 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:27.832155 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:27.832290 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.832400 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.832443 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:25:27.839416 26361 raft_consensus.cc:1261] T f66c8b14069348fba0f8559ec48e0023 P d861633065ed44d18a446645a57604de [term 1 FOLLOWER]: Refusing update from remote peer 1279e5892bf249468ca7d3a75489e6cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:27.839416 26590 raft_consensus.cc:1261] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 1 FOLLOWER]: Refusing update from remote peer 1279e5892bf249468ca7d3a75489e6cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:27.840221 26971 consensus_queue.cc:1035] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d861633065ed44d18a446645a57604de" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:27.840430 26966 consensus_queue.cc:1035] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a55f1a3a4fab4722acf53768520e944c" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 45223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:27.847883 26975 mvcc.cc:205] Tried to move back new op lower bound from 7317451275621474304 to 7317451275470917632. Current Snapshot: MvccSnapshot[applied={T|T < 7317451275621474304}]
I20260811 22:25:27.849376 26981 mvcc.cc:205] Tried to move back new op lower bound from 7317451275621474304 to 7317451275470917632. Current Snapshot: MvccSnapshot[applied={T|T < 7317451275621474304}]
I20260811 22:25:27.859297 26100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:27.859516 26100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:27.862484 26100 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecd9600e3cb84149803b6e8b21ac08d0: Sending DeleteTablet for 3 replicas of tablet f66c8b14069348fba0f8559ec48e0023
I20260811 22:25:27.863466 26804 tablet_service.cc:1514] Processing DeleteTablet for tablet f66c8b14069348fba0f8559ec48e0023 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:27 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:56858
I20260811 22:25:27.863499 26331 tablet_service.cc:1514] Processing DeleteTablet for tablet f66c8b14069348fba0f8559ec48e0023 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:27 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:56016
I20260811 22:25:27.863574 26557 tablet_service.cc:1514] Processing DeleteTablet for tablet f66c8b14069348fba0f8559ec48e0023 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:27 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:57426
I20260811 22:25:27.864121 26997 tablet_replica.cc:330] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: stopping tablet replica
I20260811 22:25:27.864486 26997 raft_consensus.cc:2229] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:27.864521 26995 tablet_replica.cc:330] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: stopping tablet replica
I20260811 22:25:27.864859 26995 raft_consensus.cc:2229] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:27.864943 26997 raft_consensus.cc:2258] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:27.865046 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 26206
I20260811 22:25:27.869828 26995 raft_consensus.cc:2258] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:27.876091 26995 ts_tablet_manager.cc:1905] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:27.879243 26995 ts_tablet_manager.cc:1918] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:27.879379 26995 log.cc:1196] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/f66c8b14069348fba0f8559ec48e0023
I20260811 22:25:27.879776 26995 ts_tablet_manager.cc:1939] T f66c8b14069348fba0f8559ec48e0023 P a55f1a3a4fab4722acf53768520e944c: Deleting consensus metadata
I20260811 22:25:27.881412 26075 catalog_manager.cc:4841] TS a55f1a3a4fab4722acf53768520e944c (127.26.125.2:45223): tablet f66c8b14069348fba0f8559ec48e0023 (table test-table [id=ee3442d47d634edb864e7e42e30d2e4a]) successfully deleted
I20260811 22:25:27.908699 26997 ts_tablet_manager.cc:1905] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:27.912528 26997 ts_tablet_manager.cc:1918] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:27.912755 26997 log.cc:1196] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/f66c8b14069348fba0f8559ec48e0023
I20260811 22:25:27.913197 26997 ts_tablet_manager.cc:1939] T f66c8b14069348fba0f8559ec48e0023 P 1279e5892bf249468ca7d3a75489e6cd: Deleting consensus metadata
I20260811 22:25:27.914677 26077 catalog_manager.cc:4841] TS 1279e5892bf249468ca7d3a75489e6cd (127.26.125.3:34801): tablet f66c8b14069348fba0f8559ec48e0023 (table test-table [id=ee3442d47d634edb864e7e42e30d2e4a]) successfully deleted
W20260811 22:25:27.915563 26075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:42785 (error 108)
I20260811 22:25:27.916635 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.916340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:42785 (local address 127.0.0.1:56024)
0811 22:25:27.916472 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:27.916544 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 22:25:27.916738 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 26457
W20260811 22:25:27.927161 26075 catalog_manager.cc:4584] TS d861633065ed44d18a446645a57604de (127.26.125.1:42785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f66c8b14069348fba0f8559ec48e0023: Network error: Client connection negotiation failed: client connection to 127.26.125.1:42785: connect: Connection refused (error 111)
I20260811 22:25:27.932372 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 26684
I20260811 22:25:27.954362 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 26043
2026-08-11T22:25:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2596 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T22:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:28Z Disabled control of system clock
I20260811 22:25:28.007710 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:43047
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:43047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:25:28.124984 27007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:28.125322 27007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:28.125362 27007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:28.129638 27007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:28.129734 27007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:28.129748 27007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:28.129767 27007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:28.129781 27007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:28.134893 27007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:43047
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:43047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:28.136108 27007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:28.137580 27007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:28.146561 27014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.146646 27017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.146659 27021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.146833 27007 server_base.cc:1034] running on GCE node
I20260811 22:25:28.147513 27007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:28.148726 27007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:28.149906 27007 hybrid_clock.cc:648] HybridClock initialized: now 1786487128149881 us; error 41 us; skew 500 ppm
I20260811 22:25:28.152009 27007 webserver.cc:460] Webserver started at http://127.26.125.62:35631/ using document root <none> and password file <none>
I20260811 22:25:28.152601 27007 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:28.152666 27007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:28.152872 27007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:28.155066 27007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "d5c0892783534681ab37743734f27fe1"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "a9991459a2d136185d156fdd854c1d12"
server_key_iv: "d7b648744c5b162e1320980f9ca7abe0"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.155619 27007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "d5c0892783534681ab37743734f27fe1"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "a9991459a2d136185d156fdd854c1d12"
server_key_iv: "d7b648744c5b162e1320980f9ca7abe0"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.159344 27007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:25:28.161842 27032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.163182 27007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:25:28.163290 27007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "d5c0892783534681ab37743734f27fe1"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "a9991459a2d136185d156fdd854c1d12"
server_key_iv: "d7b648744c5b162e1320980f9ca7abe0"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.163400 27007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:28.182158 27007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:28.182991 27007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:28.183182 27007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:28.190403 27007 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:43047
I20260811 22:25:28.190418 27119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:43047 every 8 connection(s)
I20260811 22:25:28.191483 27007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:28.194593 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27007
I20260811 22:25:28.194732 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:28.194913 27121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.195087 27124 external_mini_cluster.cc:1428] Setting key 83b33e7388fb1c32773f45f7af663738
I20260811 22:25:28.201534 27121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1: Bootstrap starting.
I20260811 22:25:28.203971 27121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.204762 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.196543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38656 (local address 127.26.125.62:43047)
0811 22:25:28.197237 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.197252 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.197289 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:25:28.198183 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.198203 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.198653 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.199006 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.199450 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.200423 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.201351 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.201887 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.202041 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.202922 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.202929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.202947 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:28.203349 (+ 402us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.203407 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:28.203427 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.203525 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.203657 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.203949 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":148,"threads_started":1}
I20260811 22:25:28.204748 27121 log.cc:824] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:28.206992 27121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1: No bootstrap required, opened a new log
I20260811 22:25:28.210597 27121 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c0892783534681ab37743734f27fe1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43047 } }
I20260811 22:25:28.210876 27121 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.210940 27121 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c0892783534681ab37743734f27fe1, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.211352 27121 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c0892783534681ab37743734f27fe1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43047 } }
I20260811 22:25:28.211472 27121 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:28.211526 27121 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:28.211601 27121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:28.212540 27121 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c0892783534681ab37743734f27fe1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43047 } }
I20260811 22:25:28.212905 27121 leader_election.cc:302] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5c0892783534681ab37743734f27fe1; no voters:
I20260811 22:25:28.213248 27121 leader_election.cc:288] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:28.213335 27141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:28.213554 27141 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [term 1 LEADER]: Becoming Leader. State: Replica: d5c0892783534681ab37743734f27fe1, State: Running, Role: LEADER
I20260811 22:25:28.213856 27141 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c0892783534681ab37743734f27fe1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43047 } }
I20260811 22:25:28.214524 27121 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:28.215842 27148 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5c0892783534681ab37743734f27fe1. Latest consensus state: current_term: 1 leader_uuid: "d5c0892783534681ab37743734f27fe1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c0892783534681ab37743734f27fe1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43047 } } }
I20260811 22:25:28.215884 27144 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5c0892783534681ab37743734f27fe1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c0892783534681ab37743734f27fe1" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43047 } } }
I20260811 22:25:28.215982 27148 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:28.215994 27144 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:28.216958 27160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:28.219833 27160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:28.225666 27160 catalog_manager.cc:1347] Generated new cluster ID: ebb84163e78d413389b27143bc3798e7
I20260811 22:25:28.225752 27160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:28.233029 27160 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:28.233989 27160 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:28.241125 27160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1: Generated new TSK 0
I20260811 22:25:28.241875 27160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:28.247138 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43047
--builtin_ntp_servers=127.26.125.20:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:25:28.369067 27180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:28.369412 27180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:28.369454 27180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:28.373836 27180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:28.373926 27180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:28.373994 27180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:25:28.379070 27180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:28.380359 27180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:28.381896 27180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:28.392891 27189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.392925 27191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.392927 27187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.392973 27180 server_base.cc:1034] running on GCE node
I20260811 22:25:28.393725 27180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:28.394523 27180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:28.395720 27180 hybrid_clock.cc:648] HybridClock initialized: now 1786487128395695 us; error 37 us; skew 500 ppm
I20260811 22:25:28.398028 27180 webserver.cc:460] Webserver started at http://127.26.125.1:36007/ using document root <none> and password file <none>
I20260811 22:25:28.398722 27180 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:28.398808 27180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:28.399088 27180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:28.401510 27180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "a02c987cb7ea42daa4571d3561e7c68b"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "b81e4e3ac8cddc6aafd585d07509bff8"
server_key_iv: "aa5f3b6f136316e65eb0117c64955e27"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.402179 27180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "a02c987cb7ea42daa4571d3561e7c68b"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "b81e4e3ac8cddc6aafd585d07509bff8"
server_key_iv: "aa5f3b6f136316e65eb0117c64955e27"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.406407 27180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:25:28.409314 27199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.410950 27180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:28.411113 27180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "a02c987cb7ea42daa4571d3561e7c68b"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "b81e4e3ac8cddc6aafd585d07509bff8"
server_key_iv: "aa5f3b6f136316e65eb0117c64955e27"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.411248 27180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:28.439405 27180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:28.440177 27180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:28.440369 27180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:28.441076 27180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:28.442154 27180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:28.442209 27180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.442267 27180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:28.442294 27180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.451611 27180 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:43977
I20260811 22:25:28.451647 27391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:43977 every 8 connection(s)
I20260811 22:25:28.452634 27180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:28.456053 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27180
I20260811 22:25:28.456194 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:28.456506 27124 external_mini_cluster.cc:1428] Setting key 92346410e2e7f64085ffaffa5f2395d2
I20260811 22:25:28.458640 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43047
--builtin_ntp_servers=127.26.125.20:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:25:28.463850 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.454787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:35773 (local address 127.26.125.62:43047)
0811 22:25:28.454965 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.454970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.456342 (+ 1372us) server_negotiation.cc:407] Connection header received
0811 22:25:28.457896 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.457898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.457942 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.458006 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.459136 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.459868 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.461444 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.461684 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.461912 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.463383 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.463387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.463389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:28.463402 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.463424 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:28.463432 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.463495 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.463666 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.463727 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 22:25:28.465220 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.455318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:43047 (local address 127.26.125.1:35773)
0811 22:25:28.456146 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.456213 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.457672 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.458246 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.458254 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.458350 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.458987 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.459005 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.459995 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.459997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.461284 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.461296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.461990 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.461995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.462231 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.463048 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:28.463050 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.463114 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:28.463210 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.463513 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.463523 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.463530 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.463662 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.464044 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":134,"threads_started":1}
I20260811 22:25:28.466605 27392 heartbeater.cc:347] Connected to a master server at 127.26.125.62:43047
I20260811 22:25:28.466910 27392 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:28.467561 27392 heartbeater.cc:511] Master 127.26.125.62:43047 requested a full tablet report, sending...
I20260811 22:25:28.469156 27060 ts_manager.cc:194] Registered new tserver with Master: a02c987cb7ea42daa4571d3561e7c68b (127.26.125.1:43977)
I20260811 22:25:28.470360 27060 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.1:35773
W20260811 22:25:28.581766 27396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:28.582100 27396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:28.582140 27396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:28.586385 27396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:28.586486 27396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:28.586548 27396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:25:28.592221 27396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:28.593554 27396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:28.595160 27396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:28.605747 27405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.605741 27404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.605731 27407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.605803 27396 server_base.cc:1034] running on GCE node
I20260811 22:25:28.606384 27396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:28.607211 27396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:28.608409 27396 hybrid_clock.cc:648] HybridClock initialized: now 1786487128608383 us; error 40 us; skew 500 ppm
I20260811 22:25:28.610523 27396 webserver.cc:460] Webserver started at http://127.26.125.2:41703/ using document root <none> and password file <none>
I20260811 22:25:28.611196 27396 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:28.611274 27396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:28.611526 27396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:28.613760 27396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "71d1eb4b109642b185b622c240794ced"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "f33de0f45b7c31b95ffb2d61521f1a2b"
server_key_iv: "da444abdb8e8294b930bb7d38ae936f5"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.614336 27396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "71d1eb4b109642b185b622c240794ced"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "f33de0f45b7c31b95ffb2d61521f1a2b"
server_key_iv: "da444abdb8e8294b930bb7d38ae936f5"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.618479 27396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260811 22:25:28.621225 27421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.622812 27396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:28.622984 27396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "71d1eb4b109642b185b622c240794ced"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "f33de0f45b7c31b95ffb2d61521f1a2b"
server_key_iv: "da444abdb8e8294b930bb7d38ae936f5"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.623106 27396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:28.644929 27396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:28.645754 27396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:28.645955 27396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:28.646610 27396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:28.647706 27396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:28.647758 27396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.647822 27396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:28.647850 27396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.657236 27396 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:43819
I20260811 22:25:28.657256 27605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:43819 every 8 connection(s)
I20260811 22:25:28.658255 27396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:28.668149 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27396
I20260811 22:25:28.668299 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:28.668613 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.660300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:45243 (local address 127.26.125.62:43047)
0811 22:25:28.660440 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.660444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.661698 (+ 1254us) server_negotiation.cc:407] Connection header received
0811 22:25:28.663334 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.663338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.663400 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.663475 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.664497 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.665029 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.666235 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.666541 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.666637 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.668049 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.668053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.668055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:28.668071 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.668096 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:28.668105 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.668178 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.668408 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.668487 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:28.668633 27124 external_mini_cluster.cc:1428] Setting key d917cade71561b9375d1074b78353001
I20260811 22:25:28.669755 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.660790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:43047 (local address 127.26.125.2:45243)
0811 22:25:28.661532 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.661578 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.663088 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.663698 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.663706 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.663809 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.664344 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.664358 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.665157 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.665161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.666106 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.666118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.666704 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.666709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.666921 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.667693 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:28.667695 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.667745 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:28.667825 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.668196 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.668207 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.668213 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.668368 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.668737 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":111,"threads_started":1}
I20260811 22:25:28.671056 27606 heartbeater.cc:347] Connected to a master server at 127.26.125.62:43047
I20260811 22:25:28.671106 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43047
--builtin_ntp_servers=127.26.125.20:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:25:28.671319 27606 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:28.671878 27606 heartbeater.cc:511] Master 127.26.125.62:43047 requested a full tablet report, sending...
I20260811 22:25:28.672909 27060 ts_manager.cc:194] Registered new tserver with Master: 71d1eb4b109642b185b622c240794ced (127.26.125.2:43819)
I20260811 22:25:28.673482 27060 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.2:45243
W20260811 22:25:28.793744 27610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:28.794099 27610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:28.794157 27610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:28.798451 27610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:28.798544 27610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:28.798627 27610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:25:28.804409 27610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:43047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:28.805809 27610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:28.807492 27610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:28.818656 27610 server_base.cc:1034] running on GCE node
W20260811 22:25:28.818605 27628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.818605 27623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.818969 27621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.819432 27610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:28.820182 27610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:28.821381 27610 hybrid_clock.cc:648] HybridClock initialized: now 1786487128821337 us; error 55 us; skew 500 ppm
I20260811 22:25:28.823541 27610 webserver.cc:460] Webserver started at http://127.26.125.3:43129/ using document root <none> and password file <none>
I20260811 22:25:28.824196 27610 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:28.824277 27610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:28.824506 27610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:28.826946 27610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "72e04b3c6c244e7b87861411bf8ae185"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "321165d41a0ac4b8b5939b48b97ff2c6"
server_key_iv: "9b5ab6e202e2678129efc64426afb820"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.827637 27610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "72e04b3c6c244e7b87861411bf8ae185"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "321165d41a0ac4b8b5939b48b97ff2c6"
server_key_iv: "9b5ab6e202e2678129efc64426afb820"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.832077 27610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:28.835034 27635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.836427 27610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:28.836565 27610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "72e04b3c6c244e7b87861411bf8ae185"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-5bn2"
server_key: "321165d41a0ac4b8b5939b48b97ff2c6"
server_key_iv: "9b5ab6e202e2678129efc64426afb820"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.836684 27610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:28.863520 27610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:28.864286 27610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:28.864477 27610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:28.865121 27610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:28.866168 27610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:28.866218 27610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.866274 27610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:28.866298 27610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.875221 27610 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:44169
I20260811 22:25:28.875245 27851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:44169 every 8 connection(s)
I20260811 22:25:28.876224 27610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:28.879647 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27610
I20260811 22:25:28.879768 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:28.880038 27124 external_mini_cluster.cc:1428] Setting key 183b4ffe3020ee929fb9b1629355d8ec
I20260811 22:25:28.886487 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.878232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:47817 (local address 127.26.125.62:43047)
0811 22:25:28.878416 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.878421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.879768 (+ 1347us) server_negotiation.cc:407] Connection header received
0811 22:25:28.881470 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.881474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.881531 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.881615 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.882794 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.883356 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.884511 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.884738 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.884811 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.885942 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.885945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.885947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:28.885961 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.885984 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:28.885994 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.886058 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.886226 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.886294 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:25:28.887389 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.878796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:43047 (local address 127.26.125.3:47817)
0811 22:25:28.879604 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.879651 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.881232 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.881848 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.881858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.881980 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.882671 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.882687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.883455 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.883457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.884398 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.884407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.884876 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.884879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.885030 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.885649 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:28.885651 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.885698 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:28.885777 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.886064 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.886069 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.886073 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.886172 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.886429 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":129,"threads_started":1}
I20260811 22:25:28.888726 27852 heartbeater.cc:347] Connected to a master server at 127.26.125.62:43047
I20260811 22:25:28.889002 27852 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:28.889622 27852 heartbeater.cc:511] Master 127.26.125.62:43047 requested a full tablet report, sending...
I20260811 22:25:28.890830 27060 ts_manager.cc:194] Registered new tserver with Master: 72e04b3c6c244e7b87861411bf8ae185 (127.26.125.3:44169)
I20260811 22:25:28.891489 27060 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.3:47817
I20260811 22:25:28.894402 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:28.900784 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.896494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.26.125.62:43047)
0811 22:25:28.896650 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.896654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.896803 (+ 149us) server_negotiation.cc:407] Connection header received
0811 22:25:28.896910 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.896913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.896957 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.897037 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.897333 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.897946 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.898920 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.899164 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.899254 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.900204 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.900207 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.900209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:28.900224 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.900248 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:28.900256 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.900370 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.900575 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.900623 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:25:28.904215 27060 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:28.907488 27060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:28.923605 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.917530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:43819 (local address 127.0.0.1:34916)
0811 22:25:28.917904 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.917961 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.918119 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.918909 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.918916 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.918943 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:28.919100 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.919108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.920321 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.920324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.921716 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.921726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.923002 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.923006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.923134 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.923171 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.923278 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.923428 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260811 22:25:28.923609 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.917686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34916 (local address 127.26.125.2:43819)
0811 22:25:28.918153 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.918164 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.918190 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:25:28.918262 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.918268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.918521 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.918699 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:28.919214 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.920194 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.921845 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.922658 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.922763 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.923040 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.923333 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.923445 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":139,"threads_started":1}
I20260811 22:25:28.924328 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.918700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:44169 (local address 127.0.0.1:42516)
0811 22:25:28.918960 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.918990 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.919079 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.920129 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.920132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.920155 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:28.920262 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.920268 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.921598 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.921601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.922699 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.922711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.923916 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.923920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.924027 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.924052 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.924105 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.924184 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":100,"threads_started":1}
I20260811 22:25:28.924541 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.918314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:43977 (local address 127.0.0.1:53560)
0811 22:25:28.918565 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.918580 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.918661 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.920366 (+ 1705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.920368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.920383 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:28.920459 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.920462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.921397 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.921400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.922754 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.922762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.924158 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.924162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.924251 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.924273 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.924347 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.924414 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":108,"threads_started":1}
I20260811 22:25:28.924990 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.918508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53560 (local address 127.26.125.1:43977)
0811 22:25:28.919645 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.919653 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.919680 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:25:28.919762 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.919768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.919978 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.920143 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:28.920572 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.921284 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.922909 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.923771 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.923881 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.924098 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.924695 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.924817 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":90,"threads_started":1}
I20260811 22:25:28.925016 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.919277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.26.125.3:44169)
0811 22:25:28.919736 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.919746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.919765 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:25:28.919826 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.919830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.919988 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.920143 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:28.920350 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.921484 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.922902 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.923771 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.924525 (+ 754us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.924728 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.924774 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.924853 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":155,"threads_started":1}
I20260811 22:25:28.927564 27493 tablet_service.cc:1467] Processing CreateTablet for tablet d3ed37c881dd4f49b0ea306a64e4f99a (DEFAULT_TABLE table=test-table [id=350299e16fc549ada8548ab37ee283cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.927764 27715 tablet_service.cc:1467] Processing CreateTablet for tablet d3ed37c881dd4f49b0ea306a64e4f99a (DEFAULT_TABLE table=test-table [id=350299e16fc549ada8548ab37ee283cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.927970 27277 tablet_service.cc:1467] Processing CreateTablet for tablet d3ed37c881dd4f49b0ea306a64e4f99a (DEFAULT_TABLE table=test-table [id=350299e16fc549ada8548ab37ee283cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.928700 27715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3ed37c881dd4f49b0ea306a64e4f99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.928745 27493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3ed37c881dd4f49b0ea306a64e4f99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.928809 27277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3ed37c881dd4f49b0ea306a64e4f99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.936604 27882 tablet_bootstrap.cc:492] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: Bootstrap starting.
I20260811 22:25:28.936748 27885 tablet_bootstrap.cc:492] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b: Bootstrap starting.
I20260811 22:25:28.938637 27882 tablet_bootstrap.cc:654] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.938638 27885 tablet_bootstrap.cc:654] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.939278 27881 tablet_bootstrap.cc:492] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: Bootstrap starting.
I20260811 22:25:28.939527 27885 log.cc:824] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:28.939620 27882 log.cc:824] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:28.941776 27881 tablet_bootstrap.cc:654] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.942057 27885 tablet_bootstrap.cc:492] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b: No bootstrap required, opened a new log
I20260811 22:25:28.942273 27885 ts_tablet_manager.cc:1397] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:28.942485 27882 tablet_bootstrap.cc:492] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: No bootstrap required, opened a new log
I20260811 22:25:28.942678 27882 ts_tablet_manager.cc:1397] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:28.942670 27881 log.cc:824] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:28.944931 27881 tablet_bootstrap.cc:492] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: No bootstrap required, opened a new log
I20260811 22:25:28.945147 27881 ts_tablet_manager.cc:1397] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:28.946249 27882 raft_consensus.cc:348] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:28.946499 27882 raft_consensus.cc:374] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.946553 27882 raft_consensus.cc:729] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d1eb4b109642b185b622c240794ced, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.946622 27885 raft_consensus.cc:348] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:28.946913 27885 raft_consensus.cc:374] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.946969 27885 raft_consensus.cc:729] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02c987cb7ea42daa4571d3561e7c68b, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.947052 27882 consensus_queue.cc:260] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:28.947427 27885 consensus_queue.cc:260] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:28.947863 27606 heartbeater.cc:503] Master 127.26.125.62:43047 was elected leader, sending a full tablet report...
I20260811 22:25:28.948247 27882 ts_tablet_manager.cc:1428] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260811 22:25:28.948449 27885 ts_tablet_manager.cc:1428] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:25:28.949489 27392 heartbeater.cc:503] Master 127.26.125.62:43047 was elected leader, sending a full tablet report...
I20260811 22:25:28.949340 27881 raft_consensus.cc:348] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:28.949592 27881 raft_consensus.cc:374] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.949640 27881 raft_consensus.cc:729] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72e04b3c6c244e7b87861411bf8ae185, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.950058 27881 consensus_queue.cc:260] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:28.950742 27852 heartbeater.cc:503] Master 127.26.125.62:43047 was elected leader, sending a full tablet report...
I20260811 22:25:28.951166 27881 ts_tablet_manager.cc:1428] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260811 22:25:28.953795 27393 tablet.cc:2369] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:29.032346 27890 raft_consensus.cc:482] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:29.032635 27890 raft_consensus.cc:504] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:29.033854 27890 leader_election.cc:288] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71d1eb4b109642b185b622c240794ced (127.26.125.2:43819), 72e04b3c6c244e7b87861411bf8ae185 (127.26.125.3:44169)
I20260811 22:25:29.037912 27893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.034155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:43819 (local address 127.26.125.1:50295)
0811 22:25:29.034434 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:29.034450 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:29.034578 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:29.035051 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:29.035055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:29.035075 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:29.035233 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:29.035240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.035986 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.035990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:29.036860 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:29.036870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.037484 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.037487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:29.037582 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.037601 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:25:29.037668 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:25:29.037738 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":23,"spinlock_wait_cycles":23424,"thread_start_us":99,"threads_started":1}
I20260811 22:25:29.037886 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.034275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:50295 (local address 127.26.125.2:43819)
0811 22:25:29.034442 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:29.034446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:29.034665 (+ 219us) server_negotiation.cc:407] Connection header received
0811 22:25:29.034749 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:29.034752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:29.034812 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:29.034919 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:29.035357 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.035872 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.036987 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.037365 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.037473 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.037529 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:29.037695 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:25:29.037766 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:25:29.038058 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.034689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:50205 (local address 127.26.125.3:44169)
0811 22:25:29.034838 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:29.034842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:29.035094 (+ 252us) server_negotiation.cc:407] Connection header received
0811 22:25:29.035209 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:29.035211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:29.035250 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:29.035321 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:29.035620 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.036115 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.037147 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.037542 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.037649 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.037697 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:29.037870 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 22:25:29.037942 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:25:29.038080 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.034563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:44169 (local address 127.26.125.1:50205)
0811 22:25:29.035008 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:29.035023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:29.035129 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:29.035365 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:29.035369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:29.035388 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:29.035506 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:29.035512 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.036223 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.036226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:29.037028 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:29.037038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.037654 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.037658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:29.037756 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.037779 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:25:29.037843 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 22:25:29.037918 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":23,"thread_start_us":135,"threads_started":1}
I20260811 22:25:29.038740 27539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3ed37c881dd4f49b0ea306a64e4f99a" candidate_uuid: "a02c987cb7ea42daa4571d3561e7c68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d1eb4b109642b185b622c240794ced" is_pre_election: true
I20260811 22:25:29.038738 27751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3ed37c881dd4f49b0ea306a64e4f99a" candidate_uuid: "a02c987cb7ea42daa4571d3561e7c68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72e04b3c6c244e7b87861411bf8ae185" is_pre_election: true
I20260811 22:25:29.039127 27539 raft_consensus.cc:2454] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02c987cb7ea42daa4571d3561e7c68b in term 0.
I20260811 22:25:29.039127 27751 raft_consensus.cc:2454] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02c987cb7ea42daa4571d3561e7c68b in term 0.
I20260811 22:25:29.039690 27208 leader_election.cc:302] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72e04b3c6c244e7b87861411bf8ae185, a02c987cb7ea42daa4571d3561e7c68b; no voters:
I20260811 22:25:29.040022 27890 raft_consensus.cc:2781] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:29.040123 27890 raft_consensus.cc:482] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:29.040181 27890 raft_consensus.cc:3037] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:29.041493 27890 raft_consensus.cc:504] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:29.041909 27890 leader_election.cc:288] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [CANDIDATE]: Term 1 election: Requested vote from peers 71d1eb4b109642b185b622c240794ced (127.26.125.2:43819), 72e04b3c6c244e7b87861411bf8ae185 (127.26.125.3:44169)
I20260811 22:25:29.042322 27751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3ed37c881dd4f49b0ea306a64e4f99a" candidate_uuid: "a02c987cb7ea42daa4571d3561e7c68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72e04b3c6c244e7b87861411bf8ae185"
I20260811 22:25:29.042420 27539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3ed37c881dd4f49b0ea306a64e4f99a" candidate_uuid: "a02c987cb7ea42daa4571d3561e7c68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d1eb4b109642b185b622c240794ced"
I20260811 22:25:29.042465 27751 raft_consensus.cc:3037] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:29.042567 27539 raft_consensus.cc:3037] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:29.043735 27751 raft_consensus.cc:2454] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02c987cb7ea42daa4571d3561e7c68b in term 1.
I20260811 22:25:29.043773 27539 raft_consensus.cc:2454] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02c987cb7ea42daa4571d3561e7c68b in term 1.
I20260811 22:25:29.044114 27208 leader_election.cc:302] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71d1eb4b109642b185b622c240794ced, a02c987cb7ea42daa4571d3561e7c68b; no voters:
I20260811 22:25:29.044319 27890 raft_consensus.cc:2781] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:29.044580 27890 raft_consensus.cc:686] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [term 1 LEADER]: Becoming Leader. State: Replica: a02c987cb7ea42daa4571d3561e7c68b, State: Running, Role: LEADER
I20260811 22:25:29.044947 27890 consensus_queue.cc:237] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } }
I20260811 22:25:29.048360 27060 catalog_manager.cc:5495] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b reported cstate change: term changed from 0 to 1, leader changed from <none> to a02c987cb7ea42daa4571d3561e7c68b (127.26.125.1). New cstate: current_term: 1 leader_uuid: "a02c987cb7ea42daa4571d3561e7c68b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a02c987cb7ea42daa4571d3561e7c68b" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 43977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 } health_report { overall_health: UNKNOWN } } }
I20260811 22:25:29.080880 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.077405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.26.125.1:43977)
0811 22:25:29.077586 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:29.077590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:29.077604 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:29.077683 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:29.077685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:29.077742 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:29.077835 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:29.078033 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.078560 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.079413 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.079655 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.079733 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.079982 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:29.080442 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:29.080564 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:29.080683 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 22:25:29.080727 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:25:29.087647 27539 raft_consensus.cc:1261] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 1 FOLLOWER]: Refusing update from remote peer a02c987cb7ea42daa4571d3561e7c68b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:29.087647 27751 raft_consensus.cc:1261] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 1 FOLLOWER]: Refusing update from remote peer a02c987cb7ea42daa4571d3561e7c68b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:29.088507 27890 consensus_queue.cc:1035] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [LEADER]: Connected to new peer: Peer: permanent_uuid: "71d1eb4b109642b185b622c240794ced" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:29.088754 27895 consensus_queue.cc:1035] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b [LEADER]: Connected to new peer: Peer: permanent_uuid: "72e04b3c6c244e7b87861411bf8ae185" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 44169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:29.099022 27902 mvcc.cc:205] Tried to move back new op lower bound from 7317451280733589504 to 7317451280570974208. Current Snapshot: MvccSnapshot[applied={T|T < 7317451280733589504}]
I20260811 22:25:29.099244 27905 mvcc.cc:205] Tried to move back new op lower bound from 7317451280733589504 to 7317451280570974208. Current Snapshot: MvccSnapshot[applied={T|T < 7317451280733589504}]
I20260811 22:25:29.106498 27058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:29.106690 27058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:29.109356 27058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5c0892783534681ab37743734f27fe1: Sending DeleteTablet for 3 replicas of tablet d3ed37c881dd4f49b0ea306a64e4f99a
I20260811 22:25:29.110285 27277 tablet_service.cc:1514] Processing DeleteTablet for tablet d3ed37c881dd4f49b0ea306a64e4f99a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:29 UTC) from {username='slave'} at 127.0.0.1:53560
I20260811 22:25:29.110285 27493 tablet_service.cc:1514] Processing DeleteTablet for tablet d3ed37c881dd4f49b0ea306a64e4f99a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:29 UTC) from {username='slave'} at 127.0.0.1:34916
I20260811 22:25:29.110448 27715 tablet_service.cc:1514] Processing DeleteTablet for tablet d3ed37c881dd4f49b0ea306a64e4f99a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:29 UTC) from {username='slave'} at 127.0.0.1:42516
I20260811 22:25:29.110939 27914 tablet_replica.cc:330] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: stopping tablet replica
I20260811 22:25:29.111335 27914 raft_consensus.cc:2229] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:29.111419 27915 tablet_replica.cc:330] T d3ed37c881dd4f49b0ea306a64e4f99a P a02c987cb7ea42daa4571d3561e7c68b: stopping tablet replica
I20260811 22:25:29.111519 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27180
I20260811 22:25:29.111713 27914 raft_consensus.cc:2258] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:29.119736 27920 tablet_replica.cc:330] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: stopping tablet replica
I20260811 22:25:29.120182 27920 raft_consensus.cc:2229] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:29.120617 27920 raft_consensus.cc:2258] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:29.121269 27914 ts_tablet_manager.cc:1905] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:29.122232 27920 ts_tablet_manager.cc:1905] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:29.124200 27914 ts_tablet_manager.cc:1918] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:29.124313 27914 log.cc:1196] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/d3ed37c881dd4f49b0ea306a64e4f99a
I20260811 22:25:29.124624 27914 ts_tablet_manager.cc:1939] T d3ed37c881dd4f49b0ea306a64e4f99a P 71d1eb4b109642b185b622c240794ced: Deleting consensus metadata
I20260811 22:25:29.125008 27920 ts_tablet_manager.cc:1918] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:29.125110 27920 log.cc:1196] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/d3ed37c881dd4f49b0ea306a64e4f99a
I20260811 22:25:29.125387 27920 ts_tablet_manager.cc:1939] T d3ed37c881dd4f49b0ea306a64e4f99a P 72e04b3c6c244e7b87861411bf8ae185: Deleting consensus metadata
W20260811 22:25:29.125919 27038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:43977 (error 108)
I20260811 22:25:29.126513 27038 catalog_manager.cc:4841] TS 72e04b3c6c244e7b87861411bf8ae185 (127.26.125.3:44169): tablet d3ed37c881dd4f49b0ea306a64e4f99a (table test-table [id=350299e16fc549ada8548ab37ee283cc]) successfully deleted
I20260811 22:25:29.126704 27038 catalog_manager.cc:4841] TS 71d1eb4b109642b185b622c240794ced (127.26.125.2:43819): tablet d3ed37c881dd4f49b0ea306a64e4f99a (table test-table [id=350299e16fc549ada8548ab37ee283cc]) successfully deleted
I20260811 22:25:29.126981 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27396
I20260811 22:25:29.127332 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.126992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:43977 (local address 127.0.0.1:53578)
0811 22:25:29.127159 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:29.127241 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:43977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:25:29.135960 27038 catalog_manager.cc:4584] TS a02c987cb7ea42daa4571d3561e7c68b (127.26.125.1:43977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3ed37c881dd4f49b0ea306a64e4f99a: Network error: Client connection negotiation failed: client connection to 127.26.125.1:43977: connect: Connection refused (error 111)
I20260811 22:25:29.143918 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27610
I20260811 22:25:29.161568 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27007
2026-08-11T22:25:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1194 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:29 dist-test-slave-5bn2 krb5kdc[27925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:29 dist-test-slave-5bn2 krb5kdc[27925](info): set up 2 sockets
Aug 11 22:25:29 dist-test-slave-5bn2 krb5kdc[27925](info): commencing operation
krb5kdc: starting...
W20260811 22:25:30.349566 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:30 dist-test-slave-5bn2 krb5kdc[27925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:25:30.484606 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40397
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:30.605424 27949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:30.605752 27949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:30.605813 27949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:30.610002 27949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:30.610121 27949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:30.610148 27949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:30.610172 27949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:30.610200 27949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:30.615911 27949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:30.617169 27949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:30.618304 27949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:30.627200 27955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:30.627357 27961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:30.627288 27956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:30.627595 27949 server_base.cc:1034] running on GCE node
I20260811 22:25:30.628244 27949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:30.629432 27949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:30.630630 27949 hybrid_clock.cc:648] HybridClock initialized: now 1786487130630609 us; error 34 us; skew 500 ppm
Aug 11 22:25:30 dist-test-slave-5bn2 krb5kdc[27925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:30.633730 27949 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:30.634709 27949 webserver.cc:460] Webserver started at http://127.26.125.62:42849/ using document root <none> and password file <none>
I20260811 22:25:30.635524 27949 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:30.635610 27949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:30.635834 27949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:30.638163 27949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "e9caac84e6174105ac1e740b73ed47a9"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-5bn2"
server_key: "f993ba7f9dc626b0d2f7b3d5f565a804"
server_key_iv: "9b7eb27df7066e1eedb2d21c027a24a5"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.638818 27949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "e9caac84e6174105ac1e740b73ed47a9"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-5bn2"
server_key: "f993ba7f9dc626b0d2f7b3d5f565a804"
server_key_iv: "9b7eb27df7066e1eedb2d21c027a24a5"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.643162 27949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:30.645927 27982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:30.647454 27949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:30.647594 27949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "e9caac84e6174105ac1e740b73ed47a9"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-5bn2"
server_key: "f993ba7f9dc626b0d2f7b3d5f565a804"
server_key_iv: "9b7eb27df7066e1eedb2d21c027a24a5"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.647708 27949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:30.671026 27949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:30.674263 27949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:30.674466 27949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:30.681879 27949 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:40397
I20260811 22:25:30.681898 28064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:40397 every 8 connection(s)
I20260811 22:25:30.682991 27949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:30.686105 28069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:30.692566 28069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9: Bootstrap starting.
I20260811 22:25:30.693076 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 27949
I20260811 22:25:30.693219 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:30.693542 27124 external_mini_cluster.cc:1428] Setting key d3b99055b7ec0c9af8dd99ffdf4f822e
I20260811 22:25:30.695621 28069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:30.696604 28069 log.cc:824] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:30.699162 28069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9: No bootstrap required, opened a new log
Aug 11 22:25:30 dist-test-slave-5bn2 krb5kdc[27925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:30.703258 28069 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9caac84e6174105ac1e740b73ed47a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40397 } }
I20260811 22:25:30.703498 28069 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:30.703573 28069 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9caac84e6174105ac1e740b73ed47a9, State: Initialized, Role: FOLLOWER
I20260811 22:25:30.704118 28069 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9caac84e6174105ac1e740b73ed47a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40397 } }
I20260811 22:25:30.704268 28069 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:30.704336 28069 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:30.704437 28069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:30.705354 28069 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9caac84e6174105ac1e740b73ed47a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40397 } }
I20260811 22:25:30.705713 28069 leader_election.cc:302] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9caac84e6174105ac1e740b73ed47a9; no voters:
I20260811 22:25:30.706108 28069 leader_election.cc:288] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:30.706192 28080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:30.706431 28080 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [term 1 LEADER]: Becoming Leader. State: Replica: e9caac84e6174105ac1e740b73ed47a9, State: Running, Role: LEADER
I20260811 22:25:30.706789 28080 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9caac84e6174105ac1e740b73ed47a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40397 } }
I20260811 22:25:30.707619 28069 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:30.708405 28086 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9caac84e6174105ac1e740b73ed47a9. Latest consensus state: current_term: 1 leader_uuid: "e9caac84e6174105ac1e740b73ed47a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9caac84e6174105ac1e740b73ed47a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40397 } } }
I20260811 22:25:30.708547 28086 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:30.708881 28081 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9caac84e6174105ac1e740b73ed47a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9caac84e6174105ac1e740b73ed47a9" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40397 } } }
I20260811 22:25:30.708983 28081 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:30.709282 28096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:30.710325 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:30.695107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43738 (local address 127.26.125.62:40397)
0811 22:25:30.695590 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:30.695613 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:30.695663 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:25:30.696603 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:30.696634 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:30.697141 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:30.697479 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:30.698323 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:30.699826 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:30.700730 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:30.701463 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:30.701629 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:30.704112 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:30.704117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:30.704132 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:30.704162 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:30.705895 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:30.706409 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:30.706416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:30.706423 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:30.706647 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:30.707073 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:30.707077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:30.707078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:30.707667 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:30.707862 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:30.708811 (+ 949us) server_negotiation.cc:299] Negotiation successful
0811 22:25:30.709170 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":117,"threads_started":1}
I20260811 22:25:30.712620 28096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:30.718937 28096 catalog_manager.cc:1347] Generated new cluster ID: 59e6c76c63b24542a8f8254dbd080144
I20260811 22:25:30.719033 28096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:30.731051 28096 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:30.732431 28096 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:30.740500 28096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9: Generated new TSK 0
I20260811 22:25:30.741266 28096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.125.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
Entry for principal kudu/127.26.125.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab.
I20260811 22:25:30.787109 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40397
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:30.919921 28124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:30.920222 28124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:30.920266 28124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:30.924304 28124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:30.924372 28124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:30.924430 28124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:25:30.929828 28124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.1.keytab
--principal=kudu/127.26.125.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:30.931003 28124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:30.932057 28124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:30.941835 28134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:30.941835 28132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:30.941829 28131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:30.942245 28124 server_base.cc:1034] running on GCE node
I20260811 22:25:30.942679 28124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:30.943385 28124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:30.944546 28124 hybrid_clock.cc:648] HybridClock initialized: now 1786487130944509 us; error 48 us; skew 500 ppm
Aug 11 22:25:30 dist-test-slave-5bn2 krb5kdc[27925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:30.947538 28124 init.cc:377] Logged in from keytab as kudu/127.26.125.1@KRBTEST.COM (short username kudu)
I20260811 22:25:30.948474 28124 webserver.cc:460] Webserver started at http://127.26.125.1:37283/ using document root <none> and password file <none>
I20260811 22:25:30.949096 28124 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:30.949172 28124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:30.949388 28124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:30.951644 28124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-5bn2"
server_key: "cdb91e954424ff49daf4fa9631acb448"
server_key_iv: "ea6f41429f63b8fd861f22063a96d488"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.952219 28124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-5bn2"
server_key: "cdb91e954424ff49daf4fa9631acb448"
server_key_iv: "ea6f41429f63b8fd861f22063a96d488"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.956089 28124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:25:30.958585 28149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:30.959838 28124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:25:30.959950 28124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-5bn2"
server_key: "cdb91e954424ff49daf4fa9631acb448"
server_key_iv: "ea6f41429f63b8fd861f22063a96d488"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.960065 28124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:30.995990 28124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:30.999135 28124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:30.999336 28124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:30.999977 28124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:31.000900 28124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:31.000952 28124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.001019 28124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:31.001044 28124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.009899 28124 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:46763
I20260811 22:25:31.009928 28339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:46763 every 8 connection(s)
I20260811 22:25:31.010958 28124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:31.017015 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 28124
I20260811 22:25:31.017155 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:31.017437 27124 external_mini_cluster.cc:1428] Setting key e79334bf6e0ed563f0ded0bc1b869e62
Aug 11 22:25:31 dist-test-slave-5bn2 krb5kdc[27925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.1@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:31.026283 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.012949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:52329 (local address 127.26.125.62:40397)
0811 22:25:31.013096 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.013099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.014050 (+ 951us) server_negotiation.cc:407] Connection header received
0811 22:25:31.015469 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.015480 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.015543 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.015623 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.017057 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.017683 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.019078 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.019313 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.019407 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.022431 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.022435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.022438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.022472 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.024077 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.024618 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.024622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.024625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.024732 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.025052 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.025055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.025057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.025308 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.025410 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.025935 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.026094 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:25:31.027091 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.013372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40397 (local address 127.26.125.1:52329)
0811 22:25:31.013872 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.013920 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.015233 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.015846 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.015855 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.016263 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.016842 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.016865 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.017824 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.017827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.018940 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.018951 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.019486 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.019491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.019690 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.020311 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.020332 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.022230 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.024204 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.024210 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.024225 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.024511 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.024819 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.024822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.024823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.024967 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.025395 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.025400 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.025694 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.025859 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.026140 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":127,"threads_started":1}
I20260811 22:25:31.028360 28344 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40397
I20260811 22:25:31.028663 28344 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.125.2@KRBTEST.COM; defaulting to no policy
I20260811 22:25:31.029315 28344 heartbeater.cc:511] Master 127.26.125.62:40397 requested a full tablet report, sending...
I20260811 22:25:31.030968 28009 ts_manager.cc:194] Registered new tserver with Master: 84c9a9ec7d3a4ee9802618f0f4a829dd (127.26.125.1:46763)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.2@KRBTEST.COM" created.
I20260811 22:25:31.032377 28009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.1@KRBTEST.COM'} at 127.26.125.1:52329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
Entry for principal kudu/127.26.125.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab.
I20260811 22:25:31.048439 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40397
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:31.166689 28354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:31.167042 28354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:31.167096 28354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:31.171373 28354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:31.171450 28354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:31.171521 28354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:25:31.176987 28354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.2.keytab
--principal=kudu/127.26.125.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:31.178174 28354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:31.179328 28354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:31.189406 28364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.189406 28365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.189407 28370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:31.189724 28354 server_base.cc:1034] running on GCE node
I20260811 22:25:31.190289 28354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:31.191043 28354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:31.192204 28354 hybrid_clock.cc:648] HybridClock initialized: now 1786487131192171 us; error 46 us; skew 500 ppm
Aug 11 22:25:31 dist-test-slave-5bn2 krb5kdc[27925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:31.195214 28354 init.cc:377] Logged in from keytab as kudu/127.26.125.2@KRBTEST.COM (short username kudu)
I20260811 22:25:31.196182 28354 webserver.cc:460] Webserver started at http://127.26.125.2:42281/ using document root <none> and password file <none>
I20260811 22:25:31.196762 28354 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:31.196833 28354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:31.197053 28354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:31.199291 28354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "0ea2c38927cf405b8de71d7286a746df"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "063260558d609f736a4f65b3251b102f"
server_key_iv: "dfc8ee28067ead0b89bc7e90f8cf395e"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.199877 28354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "0ea2c38927cf405b8de71d7286a746df"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "063260558d609f736a4f65b3251b102f"
server_key_iv: "dfc8ee28067ead0b89bc7e90f8cf395e"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.203835 28354 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:31.206360 28378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.207697 28354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:25:31.207819 28354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "0ea2c38927cf405b8de71d7286a746df"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "063260558d609f736a4f65b3251b102f"
server_key_iv: "dfc8ee28067ead0b89bc7e90f8cf395e"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.207935 28354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:31.219977 28354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:31.223479 28354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:31.223670 28354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:31.224308 28354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:31.225196 28354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:31.225244 28354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.225307 28354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:31.225332 28354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.233994 28354 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:43925
I20260811 22:25:31.234017 28582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:43925 every 8 connection(s)
I20260811 22:25:31.235064 28354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:31.236613 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 28354
I20260811 22:25:31.236729 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:31.237066 27124 external_mini_cluster.cc:1428] Setting key 2c184a7fa74ab55940654f990f313a05
Aug 11 22:25:31 dist-test-slave-5bn2 krb5kdc[27925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.2@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.125.3@KRBTEST.COM; defaulting to no policy
I20260811 22:25:31.249532 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.237013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:50145 (local address 127.26.125.62:40397)
0811 22:25:31.237172 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.237177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.238165 (+ 988us) server_negotiation.cc:407] Connection header received
0811 22:25:31.239552 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.239555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.239610 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.239686 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.241061 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.241586 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.242740 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.243016 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.243108 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.245828 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.245833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.245835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.245862 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.247325 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.247845 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.247849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.247851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.247942 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.248342 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.248345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.248347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.248554 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.248646 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.249209 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.249359 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.3@KRBTEST.COM" created.
I20260811 22:25:31.250535 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.237505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40397 (local address 127.26.125.2:50145)
0811 22:25:31.238001 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.238057 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.239330 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.239892 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.239900 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.240348 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.240917 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.240935 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.241709 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.241712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.242615 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.242626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.243180 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.243184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.243349 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.243914 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.243933 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.245650 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.247459 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.247467 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.247486 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.247746 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.248045 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.248048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.248050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.248250 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.248656 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.248664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.248963 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.249148 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.249559 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":123,"threads_started":1}
I20260811 22:25:31.251897 28583 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40397
I20260811 22:25:31.252168 28583 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:31.252859 28583 heartbeater.cc:511] Master 127.26.125.62:40397 requested a full tablet report, sending...
I20260811 22:25:31.254165 28009 ts_manager.cc:194] Registered new tserver with Master: 0ea2c38927cf405b8de71d7286a746df (127.26.125.2:43925)
I20260811 22:25:31.254755 28009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.2@KRBTEST.COM'} at 127.26.125.2:50145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
Entry for principal kudu/127.26.125.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab.
I20260811 22:25:31.269842 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40397
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:31.388702 28594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:31.389005 28594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:31.389073 28594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:31.393347 28594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:31.393435 28594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:31.393525 28594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:25:31.399253 28594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.3.keytab
--principal=kudu/127.26.125.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:31.400478 28594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:31.401602 28594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:31.412284 28602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.412446 28607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:31.412518 28594 server_base.cc:1034] running on GCE node
W20260811 22:25:31.412330 28601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:31.413117 28594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:31.413779 28594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:31.414952 28594 hybrid_clock.cc:648] HybridClock initialized: now 1786487131414916 us; error 51 us; skew 500 ppm
Aug 11 22:25:31 dist-test-slave-5bn2 krb5kdc[27925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:31.418028 28594 init.cc:377] Logged in from keytab as kudu/127.26.125.3@KRBTEST.COM (short username kudu)
I20260811 22:25:31.419034 28594 webserver.cc:460] Webserver started at http://127.26.125.3:37127/ using document root <none> and password file <none>
I20260811 22:25:31.419622 28594 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:31.419685 28594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:31.419881 28594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:31.422043 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "86110bba9f014f318c9fee84602c6fe9"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "4566e4679c7e65096dcbdcc79d416a9c"
server_key_iv: "582ddb82711a93df9bcad8659731dc27"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.422610 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "86110bba9f014f318c9fee84602c6fe9"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "4566e4679c7e65096dcbdcc79d416a9c"
server_key_iv: "582ddb82711a93df9bcad8659731dc27"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.426652 28594 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:25:31.429355 28614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.430766 28594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:31.430915 28594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "86110bba9f014f318c9fee84602c6fe9"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "4566e4679c7e65096dcbdcc79d416a9c"
server_key_iv: "582ddb82711a93df9bcad8659731dc27"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.431042 28594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:31.456094 28594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:31.459668 28594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:31.459857 28594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:31.460505 28594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:31.461396 28594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:31.461444 28594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.461488 28594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:31.461503 28594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.470599 28594 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:35591
I20260811 22:25:31.470638 28811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:35591 every 8 connection(s)
I20260811 22:25:31.471662 28594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:31.478288 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 28594
I20260811 22:25:31.478461 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:31.478746 27124 external_mini_cluster.cc:1428] Setting key 6f4cce4db6544f2347e1f6edb76b40b6
Aug 11 22:25:31 dist-test-slave-5bn2 krb5kdc[27925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.3@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:31.486122 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.473565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:36527 (local address 127.26.125.62:40397)
0811 22:25:31.473742 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.473747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.474638 (+ 891us) server_negotiation.cc:407] Connection header received
0811 22:25:31.476033 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.476036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.476088 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.476164 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.477459 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.478018 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.479331 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.479638 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.479748 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.482515 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.482518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.482520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.482547 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.483806 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.484317 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.484320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.484322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.484410 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.484836 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.484839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.484841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.485052 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.485132 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.485759 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.485959 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:25:31.486936 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.473959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40397 (local address 127.26.125.3:36527)
0811 22:25:31.474460 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.474519 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.475792 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.476331 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.476339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.476745 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.477313 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.477328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.478162 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.478165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.479214 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.479225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.479811 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.479815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.480021 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.480721 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.480738 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.482314 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.483920 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.483925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.483940 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.484223 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.484562 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.484565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.484566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.484734 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.485143 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.485149 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.485444 (+ 295us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.485646 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.485950 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":129,"threads_started":1}
I20260811 22:25:31.488164 28813 heartbeater.cc:347] Connected to a master server at 127.26.125.62:40397
I20260811 22:25:31.488410 28813 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:31.488922 28813 heartbeater.cc:511] Master 127.26.125.62:40397 requested a full tablet report, sending...
I20260811 22:25:31.489953 28009 ts_manager.cc:194] Registered new tserver with Master: 86110bba9f014f318c9fee84602c6fe9 (127.26.125.3:35591)
I20260811 22:25:31.490520 28009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.125.3@KRBTEST.COM'} at 127.26.125.3:36527
I20260811 22:25:31.492901 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:31.501751 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.494814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43754 (local address 127.26.125.62:40397)
0811 22:25:31.494947 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.494951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.495132 (+ 181us) server_negotiation.cc:407] Connection header received
0811 22:25:31.495254 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.495258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.495301 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.495364 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.496068 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.496573 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.497396 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.497610 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.497696 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.499050 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.499053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.499055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.499079 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.500149 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.500545 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.500549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.500551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.500632 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.500873 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.500877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.500879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.501115 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.501241 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.501471 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.501591 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:31.504647 28009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:31.507807 28009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:31.523221 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.518019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59498 (local address 127.26.125.1:46763)
0811 22:25:31.518411 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.518429 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.518452 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:25:31.518538 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.518543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.518737 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.518906 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.519215 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.520206 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.521521 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.522342 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.522456 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.522634 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.522953 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.523058 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":122,"threads_started":1}
I20260811 22:25:31.523234 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.516841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:35591 (local address 127.0.0.1:37852)
0811 22:25:31.517284 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.517334 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.517458 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.518263 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.518269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.518296 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.518459 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.518467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.520026 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.520029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.521535 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.521546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.522518 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.522521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.522689 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.522751 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.522886 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.523064 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":157,"thread_start_us":93,"threads_started":1}
I20260811 22:25:31.523236 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.517615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46763 (local address 127.0.0.1:59498)
0811 22:25:31.518067 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.518080 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.518162 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.519023 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.519026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.519043 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.519111 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.519117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.520326 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.520329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.521403 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.521412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.522454 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.522457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.522686 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.522747 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.522886 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.523078 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":137,"thread_start_us":75,"threads_started":1}
I20260811 22:25:31.524077 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.516909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:43925 (local address 127.0.0.1:53804)
0811 22:25:31.517588 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.517601 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.517672 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.518811 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.518814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.518830 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.518956 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.518963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.519985 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.519989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.521537 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.521546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.523669 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.523673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.523762 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.523785 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.523860 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.523928 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":123,"thread_start_us":116,"threads_started":1}
I20260811 22:25:31.524134 28838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.517465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.26.125.2:43925)
0811 22:25:31.518389 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.518401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.518424 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:25:31.518502 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.518507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.518677 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.518809 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.519081 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.519849 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.522698 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.523508 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.523654 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.523799 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.523890 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.523990 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":149,"threads_started":1}
I20260811 22:25:31.524705 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.516986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.26.125.3:35591)
0811 22:25:31.517756 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.517770 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.517792 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:25:31.517855 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.517859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.518078 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.518621 (+ 543us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.518774 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.519896 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.521665 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.522418 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.522508 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.522634 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.523678 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.524565 (+ 887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":94,"threads_started":1}
I20260811 22:25:31.527040 28222 tablet_service.cc:1467] Processing CreateTablet for tablet 6851d175b84b4202885c66135e821bf5 (DEFAULT_TABLE table=test-table [id=544511f673c244e3a665f98455894be2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:31.527040 28459 tablet_service.cc:1467] Processing CreateTablet for tablet 6851d175b84b4202885c66135e821bf5 (DEFAULT_TABLE table=test-table [id=544511f673c244e3a665f98455894be2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:31.527343 28698 tablet_service.cc:1467] Processing CreateTablet for tablet 6851d175b84b4202885c66135e821bf5 (DEFAULT_TABLE table=test-table [id=544511f673c244e3a665f98455894be2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:31.528265 28698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6851d175b84b4202885c66135e821bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:31.528260 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6851d175b84b4202885c66135e821bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:31.528260 28459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6851d175b84b4202885c66135e821bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:31.535450 28849 tablet_bootstrap.cc:492] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd: Bootstrap starting.
I20260811 22:25:31.536265 28852 tablet_bootstrap.cc:492] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: Bootstrap starting.
I20260811 22:25:31.537184 28847 tablet_bootstrap.cc:492] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: Bootstrap starting.
I20260811 22:25:31.538074 28849 tablet_bootstrap.cc:654] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:31.538499 28852 tablet_bootstrap.cc:654] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:31.539032 28849 log.cc:824] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:31.539315 28852 log.cc:824] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:31.539340 28847 tablet_bootstrap.cc:654] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:31.540431 28847 log.cc:824] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:31.541261 28852 tablet_bootstrap.cc:492] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: No bootstrap required, opened a new log
I20260811 22:25:31.541261 28849 tablet_bootstrap.cc:492] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd: No bootstrap required, opened a new log
I20260811 22:25:31.541469 28849 ts_tablet_manager.cc:1397] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:25:31.541471 28852 ts_tablet_manager.cc:1397] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:25:31.542316 28847 tablet_bootstrap.cc:492] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: No bootstrap required, opened a new log
I20260811 22:25:31.542493 28847 ts_tablet_manager.cc:1397] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:25:31.545312 28847 raft_consensus.cc:348] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.545562 28847 raft_consensus.cc:374] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:31.545643 28847 raft_consensus.cc:729] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86110bba9f014f318c9fee84602c6fe9, State: Initialized, Role: FOLLOWER
I20260811 22:25:31.545424 28852 raft_consensus.cc:348] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.545711 28852 raft_consensus.cc:374] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:31.545814 28852 raft_consensus.cc:729] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ea2c38927cf405b8de71d7286a746df, State: Initialized, Role: FOLLOWER
I20260811 22:25:31.545697 28849 raft_consensus.cc:348] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.545949 28849 raft_consensus.cc:374] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:31.546041 28849 raft_consensus.cc:729] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c9a9ec7d3a4ee9802618f0f4a829dd, State: Initialized, Role: FOLLOWER
I20260811 22:25:31.546128 28847 consensus_queue.cc:260] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.546289 28852 consensus_queue.cc:260] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.546476 28849 consensus_queue.cc:260] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.546882 28813 heartbeater.cc:503] Master 127.26.125.62:40397 was elected leader, sending a full tablet report...
I20260811 22:25:31.547214 28847 ts_tablet_manager.cc:1428] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:25:31.547288 28852 ts_tablet_manager.cc:1428] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:25:31.547395 28849 ts_tablet_manager.cc:1428] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:25:31.547657 28583 heartbeater.cc:503] Master 127.26.125.62:40397 was elected leader, sending a full tablet report...
I20260811 22:25:31.547672 28344 heartbeater.cc:503] Master 127.26.125.62:40397 was elected leader, sending a full tablet report...
I20260811 22:25:31.574105 28861 raft_consensus.cc:482] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:31.574499 28861 raft_consensus.cc:504] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.575997 28861 leader_election.cc:288] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86110bba9f014f318c9fee84602c6fe9 (127.26.125.3:35591), 0ea2c38927cf405b8de71d7286a746df (127.26.125.2:43925)
I20260811 22:25:31.581038 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.576302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:41257 (local address 127.26.125.3:35591)
0811 22:25:31.576443 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.576448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.576634 (+ 186us) server_negotiation.cc:407] Connection header received
0811 22:25:31.576758 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.576761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.576813 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.576895 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.577176 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.577946 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.579697 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.580327 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.580456 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.580517 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.580817 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.580893 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:25:31.581035 28838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.576699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:39445 (local address 127.26.125.2:43925)
0811 22:25:31.576813 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.576817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.576864 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:25:31.576958 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.576961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.577007 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.577075 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.577315 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.578030 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.579697 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.580327 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.580456 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.580557 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.580815 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.580893 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:25:31.581112 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.576218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:35591 (local address 127.26.125.1:41257)
0811 22:25:31.576557 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.576571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.576675 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.576919 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.576922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.576939 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.577069 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.577076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.578067 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.578069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.579572 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.579585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.580458 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.580462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.580655 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.580710 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.580789 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.580886 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":135,"thread_start_us":158,"threads_started":1}
I20260811 22:25:31.581486 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.576251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:43925 (local address 127.26.125.1:39445)
0811 22:25:31.576815 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.576824 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.576884 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.577097 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.577100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.577115 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.577223 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.577228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.578125 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.578129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.579590 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.579597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.580458 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.580462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.580653 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.580709 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.580789 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.580886 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":154,"thread_start_us":119,"threads_started":1}
I20260811 22:25:31.581847 28737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6851d175b84b4202885c66135e821bf5" candidate_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86110bba9f014f318c9fee84602c6fe9" is_pre_election: true
I20260811 22:25:31.582139 28506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6851d175b84b4202885c66135e821bf5" candidate_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ea2c38927cf405b8de71d7286a746df" is_pre_election: true
I20260811 22:25:31.582232 28737 raft_consensus.cc:2454] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c9a9ec7d3a4ee9802618f0f4a829dd in term 0.
I20260811 22:25:31.582441 28506 raft_consensus.cc:2454] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c9a9ec7d3a4ee9802618f0f4a829dd in term 0.
I20260811 22:25:31.582756 28151 leader_election.cc:302] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c9a9ec7d3a4ee9802618f0f4a829dd, 86110bba9f014f318c9fee84602c6fe9; no voters:
I20260811 22:25:31.583081 28861 raft_consensus.cc:2781] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:31.583168 28861 raft_consensus.cc:482] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:31.583200 28861 raft_consensus.cc:3037] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:31.584280 28861 raft_consensus.cc:504] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.584635 28861 leader_election.cc:288] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [CANDIDATE]: Term 1 election: Requested vote from peers 86110bba9f014f318c9fee84602c6fe9 (127.26.125.3:35591), 0ea2c38927cf405b8de71d7286a746df (127.26.125.2:43925)
I20260811 22:25:31.584954 28737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6851d175b84b4202885c66135e821bf5" candidate_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86110bba9f014f318c9fee84602c6fe9"
I20260811 22:25:31.585011 28506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6851d175b84b4202885c66135e821bf5" candidate_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ea2c38927cf405b8de71d7286a746df"
I20260811 22:25:31.585076 28737 raft_consensus.cc:3037] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:31.585131 28506 raft_consensus.cc:3037] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:31.586130 28506 raft_consensus.cc:2454] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c9a9ec7d3a4ee9802618f0f4a829dd in term 1.
I20260811 22:25:31.586261 28737 raft_consensus.cc:2454] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c9a9ec7d3a4ee9802618f0f4a829dd in term 1.
I20260811 22:25:31.586418 28152 leader_election.cc:302] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ea2c38927cf405b8de71d7286a746df, 84c9a9ec7d3a4ee9802618f0f4a829dd; no voters:
I20260811 22:25:31.586596 28861 raft_consensus.cc:2781] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:31.586818 28861 raft_consensus.cc:686] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [term 1 LEADER]: Becoming Leader. State: Replica: 84c9a9ec7d3a4ee9802618f0f4a829dd, State: Running, Role: LEADER
I20260811 22:25:31.587182 28861 consensus_queue.cc:237] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } }
I20260811 22:25:31.590668 28009 catalog_manager.cc:5495] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c9a9ec7d3a4ee9802618f0f4a829dd (127.26.125.1). New cstate: current_term: 1 leader_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c9a9ec7d3a4ee9802618f0f4a829dd" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 46763 } health_report { overall_health: HEALTHY } } }
I20260811 22:25:31.612501 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.608800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59510 (local address 127.26.125.1:46763)
0811 22:25:31.608945 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.608950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.608967 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:25:31.609104 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.609106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.609166 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.609260 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:31.609531 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.610113 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.610968 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.611218 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.611298 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.611620 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:31.612071 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:31.612189 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.612298 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.612360 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:25:31.618678 28506 raft_consensus.cc:1261] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 1 FOLLOWER]: Refusing update from remote peer 84c9a9ec7d3a4ee9802618f0f4a829dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:31.618678 28737 raft_consensus.cc:1261] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 1 FOLLOWER]: Refusing update from remote peer 84c9a9ec7d3a4ee9802618f0f4a829dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:31.619462 28861 consensus_queue.cc:1035] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ea2c38927cf405b8de71d7286a746df" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:31.619661 28865 consensus_queue.cc:1035] T 6851d175b84b4202885c66135e821bf5 P 84c9a9ec7d3a4ee9802618f0f4a829dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "86110bba9f014f318c9fee84602c6fe9" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:31.636895 28008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:31.637075 28008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:31.639443 28008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9caac84e6174105ac1e740b73ed47a9: Sending DeleteTablet for 3 replicas of tablet 6851d175b84b4202885c66135e821bf5
I20260811 22:25:31.640332 28698 tablet_service.cc:1514] Processing DeleteTablet for tablet 6851d175b84b4202885c66135e821bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:31 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:37852
I20260811 22:25:31.640331 28222 tablet_service.cc:1514] Processing DeleteTablet for tablet 6851d175b84b4202885c66135e821bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:31 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:59498
I20260811 22:25:31.640343 28459 tablet_service.cc:1514] Processing DeleteTablet for tablet 6851d175b84b4202885c66135e821bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:31 UTC) from {username='kudu', principal='kudu/127.26.125.62@KRBTEST.COM'} at 127.0.0.1:53804
I20260811 22:25:31.641125 28886 tablet_replica.cc:330] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: stopping tablet replica
I20260811 22:25:31.641580 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 28124
I20260811 22:25:31.665412 28886 raft_consensus.cc:2229] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:31.665910 28886 raft_consensus.cc:2258] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:31.668277 28886 ts_tablet_manager.cc:1905] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:31.670768 28886 ts_tablet_manager.cc:1918] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:31.670913 28886 log.cc:1196] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/6851d175b84b4202885c66135e821bf5
I20260811 22:25:31.671283 28886 ts_tablet_manager.cc:1939] T 6851d175b84b4202885c66135e821bf5 P 0ea2c38927cf405b8de71d7286a746df: Deleting consensus metadata
I20260811 22:25:31.671267 28889 tablet_replica.cc:330] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: stopping tablet replica
I20260811 22:25:31.671670 28889 raft_consensus.cc:2229] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:31.672111 28889 raft_consensus.cc:2258] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:31.673887 28889 ts_tablet_manager.cc:1905] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:31.674525 27987 catalog_manager.cc:4841] TS 0ea2c38927cf405b8de71d7286a746df (127.26.125.2:43925): tablet 6851d175b84b4202885c66135e821bf5 (table test-table [id=544511f673c244e3a665f98455894be2]) successfully deleted
W20260811 22:25:31.676970 27985 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:46763 (error 108)
I20260811 22:25:31.677253 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 28354
I20260811 22:25:31.676776 28889 ts_tablet_manager.cc:1918] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:31.677592 28889 log.cc:1196] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/6851d175b84b4202885c66135e821bf5
I20260811 22:25:31.680430 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.677838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:46763 (local address 127.0.0.1:59518)
0811 22:25:31.680199 (+ 2361us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.680310 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2240}
W20260811 22:25:31.680688 27985 catalog_manager.cc:4584] TS 84c9a9ec7d3a4ee9802618f0f4a829dd (127.26.125.1:46763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6851d175b84b4202885c66135e821bf5: Network error: Client connection negotiation failed: client connection to 127.26.125.1:46763: connect: Connection refused (error 111)
I20260811 22:25:31.690670 28889 ts_tablet_manager.cc:1939] T 6851d175b84b4202885c66135e821bf5 P 86110bba9f014f318c9fee84602c6fe9: Deleting consensus metadata
I20260811 22:25:31.692478 27985 catalog_manager.cc:4841] TS 86110bba9f014f318c9fee84602c6fe9 (127.26.125.3:35591): tablet 6851d175b84b4202885c66135e821bf5 (table test-table [id=544511f673c244e3a665f98455894be2]) successfully deleted
I20260811 22:25:31.695812 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 28594
I20260811 22:25:31.711112 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 27949
2026-08-11T22:25:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2546 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T22:25:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:31Z Disabled control of system clock
I20260811 22:25:31.747932 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:42031
--webserver_interface=127.26.125.62
--webserver_port=0
--builtin_ntp_servers=127.26.125.20:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:42031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:25:31.864377 28898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:31.864684 28898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:31.864720 28898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:31.868932 28898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:31.869004 28898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:31.869017 28898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:31.869033 28898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:31.869047 28898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:31.874142 28898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:42031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:42031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:31.875388 28898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:31.876852 28898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:31.887087 28911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.887099 28904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.887254 28907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:31.887527 28898 server_base.cc:1034] running on GCE node
I20260811 22:25:31.888273 28898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:31.889544 28898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:31.890730 28898 hybrid_clock.cc:648] HybridClock initialized: now 1786487131890699 us; error 47 us; skew 500 ppm
I20260811 22:25:31.897818 28898 webserver.cc:460] Webserver started at http://127.26.125.62:40231/ using document root <none> and password file <none>
I20260811 22:25:31.898630 28898 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:31.898734 28898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:31.899061 28898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:31.901474 28898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "cb8305a4a2db4657aabbe7a89d404db2"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "ab8487c5a7d35d56d27dca190f154a0c"
server_key_iv: "d65d6c467b7457d913c922d98f8beb85"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.902140 28898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "cb8305a4a2db4657aabbe7a89d404db2"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "ab8487c5a7d35d56d27dca190f154a0c"
server_key_iv: "d65d6c467b7457d913c922d98f8beb85"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.907245 28898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 22:25:31.910337 28920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.911978 28898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:31.912143 28898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "cb8305a4a2db4657aabbe7a89d404db2"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-5bn2"
server_key: "ab8487c5a7d35d56d27dca190f154a0c"
server_key_iv: "d65d6c467b7457d913c922d98f8beb85"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.912269 28898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:31.937687 28898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:31.938482 28898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:31.938678 28898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:31.945606 28898 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:42031
I20260811 22:25:31.945648 29012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:42031 every 8 connection(s)
I20260811 22:25:31.946637 28898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:31.950100 29013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:31.955619 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 28898
I20260811 22:25:31.955736 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:31.956015 27124 external_mini_cluster.cc:1428] Setting key 81aeadef8df9777cf857e033253f6026
I20260811 22:25:31.956235 29013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2: Bootstrap starting.
I20260811 22:25:31.958992 29013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:31.959811 29013 log.cc:824] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:31.962179 29013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2: No bootstrap required, opened a new log
I20260811 22:25:31.966238 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.957492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.26.125.62:42031)
0811 22:25:31.958190 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.958206 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.958246 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:25:31.958994 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.959016 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.959521 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.959828 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.960245 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.961386 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.962640 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.963190 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.963319 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.964199 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.964205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.964221 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:31.964679 (+ 458us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.964741 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:31.964763 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.964840 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.965047 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.965357 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":139,"threads_started":1}
I20260811 22:25:31.966329 29013 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8305a4a2db4657aabbe7a89d404db2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 42031 } }
I20260811 22:25:31.966632 29013 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:31.966696 29013 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb8305a4a2db4657aabbe7a89d404db2, State: Initialized, Role: FOLLOWER
I20260811 22:25:31.967206 29013 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8305a4a2db4657aabbe7a89d404db2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 42031 } }
I20260811 22:25:31.967355 29013 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:31.967427 29013 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:31.967530 29013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:31.968537 29013 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8305a4a2db4657aabbe7a89d404db2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 42031 } }
I20260811 22:25:31.968920 29013 leader_election.cc:302] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb8305a4a2db4657aabbe7a89d404db2; no voters:
I20260811 22:25:31.969269 29013 leader_election.cc:288] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:31.969377 29024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:31.969635 29024 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [term 1 LEADER]: Becoming Leader. State: Replica: cb8305a4a2db4657aabbe7a89d404db2, State: Running, Role: LEADER
I20260811 22:25:31.969959 29024 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8305a4a2db4657aabbe7a89d404db2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 42031 } }
I20260811 22:25:31.970593 29013 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:31.971856 29026 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb8305a4a2db4657aabbe7a89d404db2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8305a4a2db4657aabbe7a89d404db2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 42031 } } }
I20260811 22:25:31.971983 29026 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:31.972215 29028 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb8305a4a2db4657aabbe7a89d404db2. Latest consensus state: current_term: 1 leader_uuid: "cb8305a4a2db4657aabbe7a89d404db2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8305a4a2db4657aabbe7a89d404db2" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 42031 } } }
I20260811 22:25:31.972321 29028 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:31.972463 29038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:31.975584 29038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:31.981232 29038 catalog_manager.cc:1347] Generated new cluster ID: 6a6200c75e67406ea56741ab452496f0
I20260811 22:25:31.981328 29038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:31.998296 29038 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:31.999397 29038 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:32.013502 29038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2: Generated new TSK 0
I20260811 22:25:32.014521 29038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:32.025554 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.1:0
--local_ip_for_outbound_sockets=127.26.125.1
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:42031
--builtin_ntp_servers=127.26.125.20:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:25:32.149930 29060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:32.150269 29060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:32.150398 29060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:32.154547 29060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:32.154625 29060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:32.154716 29060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.1
I20260811 22:25:32.159663 29060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.125.1
--webserver_port=0
--tserver_master_addrs=127.26.125.62:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.1
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:32.160841 29060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:32.162271 29060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:32.172298 29066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:32.172326 29067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:32.172326 29070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:32.172659 29060 server_base.cc:1034] running on GCE node
I20260811 22:25:32.173095 29060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:32.173780 29060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:32.174948 29060 hybrid_clock.cc:648] HybridClock initialized: now 1786487132174914 us; error 40 us; skew 500 ppm
I20260811 22:25:32.177030 29060 webserver.cc:460] Webserver started at http://127.26.125.1:37085/ using document root <none> and password file <none>
I20260811 22:25:32.177645 29060 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:32.177716 29060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:32.177927 29060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:32.180115 29060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/data/instance:
uuid: "0fa3d5efe1964a8a997e3a5205d78850"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "6cffbde3d66ec8cbdbd78be2a21fc7c6"
server_key_iv: "08d773accbada01ca5540ae39bdc2380"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.180668 29060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance:
uuid: "0fa3d5efe1964a8a997e3a5205d78850"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "6cffbde3d66ec8cbdbd78be2a21fc7c6"
server_key_iv: "08d773accbada01ca5540ae39bdc2380"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.184581 29060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:32.187292 29079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.188647 29060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:32.188772 29060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/wal
uuid: "0fa3d5efe1964a8a997e3a5205d78850"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "6cffbde3d66ec8cbdbd78be2a21fc7c6"
server_key_iv: "08d773accbada01ca5540ae39bdc2380"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.188889 29060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:32.212751 29060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:32.213519 29060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:32.213699 29060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:32.214320 29060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:32.215351 29060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:32.215401 29060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.215456 29060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:32.215489 29060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.224421 29060 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.1:42671
I20260811 22:25:32.224444 29265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.1:42671 every 8 connection(s)
I20260811 22:25:32.225381 29060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:32.233954 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 29060
I20260811 22:25:32.234083 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:32.234364 27124 external_mini_cluster.cc:1428] Setting key 46d597c9fc44e2e1f1fda1c88835edec
I20260811 22:25:32.234983 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.227370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:36077 (local address 127.26.125.62:42031)
0811 22:25:32.227524 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.227529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.228744 (+ 1215us) server_negotiation.cc:407] Connection header received
0811 22:25:32.230186 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.230189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.230240 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.230324 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:32.231286 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.231862 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.232957 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.233172 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.233248 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.234490 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:32.234495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:32.234498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:32.234515 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:32.234540 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:32.234552 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:32.234629 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.234765 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.234826 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:25:32.235994 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.227852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:42031 (local address 127.26.125.1:36077)
0811 22:25:32.228585 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.228632 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.229936 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.230506 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.230514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.230605 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:32.231155 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.231169 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.231961 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.231963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.232851 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.232860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.233307 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.233311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.233462 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.234185 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:32.234187 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:32.234229 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:32.234297 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:32.234632 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:32.234640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:32.234645 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.234746 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.235073 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":117,"threads_started":1}
I20260811 22:25:32.236454 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.2:0
--local_ip_for_outbound_sockets=127.26.125.2
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:42031
--builtin_ntp_servers=127.26.125.20:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:25:32.237268 29267 heartbeater.cc:347] Connected to a master server at 127.26.125.62:42031
I20260811 22:25:32.237543 29267 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:32.238147 29267 heartbeater.cc:511] Master 127.26.125.62:42031 requested a full tablet report, sending...
I20260811 22:25:32.239876 28956 ts_manager.cc:194] Registered new tserver with Master: 0fa3d5efe1964a8a997e3a5205d78850 (127.26.125.1:42671)
I20260811 22:25:32.241406 28956 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.1:36077
W20260811 22:25:32.355759 29270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:32.356050 29270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:32.356089 29270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:32.360082 29270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:32.360154 29270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:32.360211 29270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.2
I20260811 22:25:32.365120 29270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.125.2
--webserver_port=0
--tserver_master_addrs=127.26.125.62:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.2
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:32.366291 29270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:32.368075 29270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:32.378799 29278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:32.378803 29279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:32.378791 29283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:32.378881 29270 server_base.cc:1034] running on GCE node
I20260811 22:25:32.379590 29270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:32.380296 29270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:32.381458 29270 hybrid_clock.cc:648] HybridClock initialized: now 1786487132381436 us; error 35 us; skew 500 ppm
I20260811 22:25:32.383458 29270 webserver.cc:460] Webserver started at http://127.26.125.2:43519/ using document root <none> and password file <none>
I20260811 22:25:32.384095 29270 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:32.384168 29270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:32.384387 29270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:32.386610 29270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/data/instance:
uuid: "8e365d07bbe9444e888b5c38d2d63f1e"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "2516be69518ff44a1cdbd5aa68b8cdbc"
server_key_iv: "f560f888c767787f8882d86e7e9cb587"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.387261 29270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance:
uuid: "8e365d07bbe9444e888b5c38d2d63f1e"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "2516be69518ff44a1cdbd5aa68b8cdbc"
server_key_iv: "f560f888c767787f8882d86e7e9cb587"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.391304 29270 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:32.394250 29299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.395797 29270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:32.395941 29270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/wal
uuid: "8e365d07bbe9444e888b5c38d2d63f1e"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "2516be69518ff44a1cdbd5aa68b8cdbc"
server_key_iv: "f560f888c767787f8882d86e7e9cb587"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.396064 29270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:32.411926 29270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:32.412714 29270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:32.412891 29270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:32.413503 29270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:32.414474 29270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:32.414521 29270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.414566 29270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:32.414594 29270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.423187 29270 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.2:46731
I20260811 22:25:32.423203 29504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.2:46731 every 8 connection(s)
I20260811 22:25:32.424196 29270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:32.425187 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 29270
I20260811 22:25:32.425297 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:32.425578 27124 external_mini_cluster.cc:1428] Setting key 0f3c94437ba5de6036f1ff804292e796
I20260811 22:25:32.428002 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.125.3:0
--local_ip_for_outbound_sockets=127.26.125.3
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:42031
--builtin_ntp_servers=127.26.125.20:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:25:32.434900 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.426519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.2:46917 (local address 127.26.125.62:42031)
0811 22:25:32.426685 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.426690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.427776 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 22:25:32.429262 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.429265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.429324 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.429406 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:32.430457 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.431048 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.432465 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.432759 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.432865 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.434245 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:32.434248 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:32.434250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:32.434265 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:32.434292 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:32.434301 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:32.434364 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.434617 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.434700 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:25:32.435858 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.426795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:42031 (local address 127.26.125.2:46917)
0811 22:25:32.427624 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.427670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.429034 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.429611 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.429618 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.429718 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:32.430297 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.430314 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.431200 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.431203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.432266 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.432278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.432945 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.432950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.433154 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.433901 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:32.433903 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:32.433963 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:32.434049 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:32.434443 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:32.434454 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:32.434460 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.434595 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.434980 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":142,"threads_started":1}
I20260811 22:25:32.437098 29506 heartbeater.cc:347] Connected to a master server at 127.26.125.62:42031
I20260811 22:25:32.437398 29506 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:32.438140 29506 heartbeater.cc:511] Master 127.26.125.62:42031 requested a full tablet report, sending...
I20260811 22:25:32.439513 28956 ts_manager.cc:194] Registered new tserver with Master: 8e365d07bbe9444e888b5c38d2d63f1e (127.26.125.2:46731)
I20260811 22:25:32.440338 28956 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.2:46917
W20260811 22:25:32.549953 29510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:32.550304 29510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:32.550345 29510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:32.554662 29510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:32.554760 29510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:32.554840 29510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.125.3
I20260811 22:25:32.559924 29510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.125.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.125.3
--webserver_port=0
--tserver_master_addrs=127.26.125.62:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.125.3
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:32.561153 29510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:32.562666 29510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:32.573206 29522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:32.573393 29525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:32.573184 29520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:32.573552 29510 server_base.cc:1034] running on GCE node
I20260811 22:25:32.574028 29510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:32.574744 29510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:32.575913 29510 hybrid_clock.cc:648] HybridClock initialized: now 1786487132575879 us; error 51 us; skew 500 ppm
I20260811 22:25:32.578485 29510 webserver.cc:460] Webserver started at http://127.26.125.3:39069/ using document root <none> and password file <none>
I20260811 22:25:32.579336 29510 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:32.579438 29510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:32.579680 29510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:32.581970 29510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/data/instance:
uuid: "0b9a2d66e9214bd79d8a64c8a18d2923"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "67bbd5ca786dc05974fffbde49497bd7"
server_key_iv: "898b93609eb154361511cfe5198ab0ef"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.582587 29510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance:
uuid: "0b9a2d66e9214bd79d8a64c8a18d2923"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "67bbd5ca786dc05974fffbde49497bd7"
server_key_iv: "898b93609eb154361511cfe5198ab0ef"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.586723 29510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:25:32.589491 29536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.590945 29510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:32.591082 29510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/wal
uuid: "0b9a2d66e9214bd79d8a64c8a18d2923"
format_stamp: "Formatted at 2026-08-11 22:25:32 on dist-test-slave-5bn2"
server_key: "67bbd5ca786dc05974fffbde49497bd7"
server_key_iv: "898b93609eb154361511cfe5198ab0ef"
server_key_version: "encryptionkey@0"
I20260811 22:25:32.591202 29510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:32.610971 29510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:32.611755 29510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:32.611970 29510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:32.612627 29510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:32.613682 29510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:32.613734 29510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.613791 29510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:32.613818 29510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:32.622943 29510 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.3:40203
I20260811 22:25:32.622961 29750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.3:40203 every 8 connection(s)
I20260811 22:25:32.623944 29510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:32.626116 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 29510
I20260811 22:25:32.626235 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:32.626503 27124 external_mini_cluster.cc:1428] Setting key 4d91ffe05247ea735ed5d1f4636351fd
I20260811 22:25:32.633873 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.625961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.3:32791 (local address 127.26.125.62:42031)
0811 22:25:32.626135 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.626140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.627403 (+ 1263us) server_negotiation.cc:407] Connection header received
0811 22:25:32.628874 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.628877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.628942 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.629017 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:32.630019 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.630573 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.631833 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.632074 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.632168 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.633334 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:32.633338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:32.633340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:32.633353 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:32.633373 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:32.633381 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:32.633442 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.633640 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.633713 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:25:32.634883 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.626430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:42031 (local address 127.26.125.3:32791)
0811 22:25:32.627234 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.627291 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.628659 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.629223 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.629230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.629324 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:32.629870 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.629885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.630710 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.630713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.631704 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.631716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.632235 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.632240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.632417 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.633033 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:32.633034 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:32.633083 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:32.633158 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:32.633454 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:32.633463 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:32.633470 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.633625 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.633969 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":131,"threads_started":1}
I20260811 22:25:32.636260 29751 heartbeater.cc:347] Connected to a master server at 127.26.125.62:42031
I20260811 22:25:32.636550 29751 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:32.637202 29751 heartbeater.cc:511] Master 127.26.125.62:42031 requested a full tablet report, sending...
I20260811 22:25:32.638414 28956 ts_manager.cc:194] Registered new tserver with Master: 0b9a2d66e9214bd79d8a64c8a18d2923 (127.26.125.3:40203)
I20260811 22:25:32.639123 28956 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.125.3:32791
I20260811 22:25:32.640415 27124 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:32.646114 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.642636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.26.125.62:42031)
0811 22:25:32.642755 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.642758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.642994 (+ 236us) server_negotiation.cc:407] Connection header received
0811 22:25:32.643091 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.643093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.643140 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.643211 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:32.643433 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.643978 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.644840 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.645055 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.645135 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.645784 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:32.645786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:32.645788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:32.645799 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:32.645817 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:32.645825 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:32.645878 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.645936 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.645987 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:25:32.649471 28956 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:32.652895 28956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:32.669768 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.663080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46731 (local address 127.0.0.1:51426)
0811 22:25:32.663747 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.663808 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.663943 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.664668 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.664675 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.664702 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:32.664867 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.664874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.666536 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.666539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.668027 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.668040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.669139 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.669143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.669216 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.669293 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.669395 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.669589 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":31,"thread_start_us":89,"threads_started":1}
I20260811 22:25:32.669831 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.664341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.26.125.1:42671)
0811 22:25:32.664978 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.664990 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.665013 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:25:32.665084 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.665088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.665262 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.665447 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:32.665613 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.666677 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.668083 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.668942 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.669306 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.669516 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.669574 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.669683 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":132,"threads_started":1}
I20260811 22:25:32.669785 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.663932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:42671 (local address 127.0.0.1:58296)
0811 22:25:32.664635 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.664648 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.664716 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.665420 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.665423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.665441 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:32.665518 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.665524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.666792 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.666795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.667898 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.667913 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.669107 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.669111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.669201 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.669315 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.669406 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.669615 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":74,"thread_start_us":99,"threads_started":1}
I20260811 22:25:32.670117 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.663577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.26.125.3:40203)
0811 22:25:32.664425 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.664435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.664464 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:25:32.664662 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.664669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.664874 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.665035 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:32.665390 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.666251 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.668319 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.668894 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.668999 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.669245 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.669739 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.669857 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":173,"threads_started":1}
I20260811 22:25:32.670536 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.663205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:40203 (local address 127.0.0.1:53410)
0811 22:25:32.664393 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.664406 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.664496 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.665189 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.665192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.665207 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:32.665279 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.665283 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.666382 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.666386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.668200 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.668208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.669028 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.669032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.669447 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.669470 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.669536 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.669650 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"mutex_wait_us":313,"thread_start_us":112,"threads_started":1}
I20260811 22:25:32.670624 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.663209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.26.125.2:46731)
0811 22:25:32.664105 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.664120 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.664150 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:25:32.664232 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.664238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.664477 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.665039 (+ 562us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:32.665198 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.666416 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.668258 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.669018 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.670073 (+ 1055us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.670305 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.670362 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.670467 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":110,"threads_started":1}
I20260811 22:25:32.673663 29628 tablet_service.cc:1467] Processing CreateTablet for tablet 8a68f71083804963be884be898806d39 (DEFAULT_TABLE table=test-table [id=15d92dfccee44d6b9f03edae2c74492c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:32.673761 29157 tablet_service.cc:1467] Processing CreateTablet for tablet 8a68f71083804963be884be898806d39 (DEFAULT_TABLE table=test-table [id=15d92dfccee44d6b9f03edae2c74492c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:32.673763 29384 tablet_service.cc:1467] Processing CreateTablet for tablet 8a68f71083804963be884be898806d39 (DEFAULT_TABLE table=test-table [id=15d92dfccee44d6b9f03edae2c74492c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:32.674778 29628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a68f71083804963be884be898806d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.674840 29384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a68f71083804963be884be898806d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.674840 29157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a68f71083804963be884be898806d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.682562 29782 tablet_bootstrap.cc:492] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850: Bootstrap starting.
I20260811 22:25:32.682562 29781 tablet_bootstrap.cc:492] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: Bootstrap starting.
I20260811 22:25:32.683840 29778 tablet_bootstrap.cc:492] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: Bootstrap starting.
I20260811 22:25:32.685170 29782 tablet_bootstrap.cc:654] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.685170 29781 tablet_bootstrap.cc:654] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.685765 29778 tablet_bootstrap.cc:654] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.686156 29781 log.cc:824] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:32.686156 29782 log.cc:824] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:32.686617 29778 log.cc:824] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:32.688623 29782 tablet_bootstrap.cc:492] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850: No bootstrap required, opened a new log
I20260811 22:25:32.688623 29781 tablet_bootstrap.cc:492] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: No bootstrap required, opened a new log
I20260811 22:25:32.688643 29778 tablet_bootstrap.cc:492] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: No bootstrap required, opened a new log
I20260811 22:25:32.688855 29778 ts_tablet_manager.cc:1397] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:25:32.688858 29781 ts_tablet_manager.cc:1397] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:25:32.688858 29782 ts_tablet_manager.cc:1397] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:32.692629 29778 raft_consensus.cc:348] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.692915 29778 raft_consensus.cc:374] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.692971 29778 raft_consensus.cc:729] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b9a2d66e9214bd79d8a64c8a18d2923, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.693251 29782 raft_consensus.cc:348] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.693251 29781 raft_consensus.cc:348] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.693540 29782 raft_consensus.cc:374] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.693540 29781 raft_consensus.cc:374] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.693591 29782 raft_consensus.cc:729] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa3d5efe1964a8a997e3a5205d78850, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.693598 29781 raft_consensus.cc:729] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e365d07bbe9444e888b5c38d2d63f1e, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.693730 29778 consensus_queue.cc:260] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.694068 29781 consensus_queue.cc:260] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.694068 29782 consensus_queue.cc:260] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.695098 29778 ts_tablet_manager.cc:1428] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:25:32.695108 29781 ts_tablet_manager.cc:1428] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 22:25:32.695108 29782 ts_tablet_manager.cc:1428] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 22:25:32.695477 29506 heartbeater.cc:503] Master 127.26.125.62:42031 was elected leader, sending a full tablet report...
I20260811 22:25:32.695479 29751 heartbeater.cc:503] Master 127.26.125.62:42031 was elected leader, sending a full tablet report...
I20260811 22:25:32.696062 29267 heartbeater.cc:503] Master 127.26.125.62:42031 was elected leader, sending a full tablet report...
I20260811 22:25:32.708508 29793 raft_consensus.cc:482] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:32.708727 29793 raft_consensus.cc:504] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.709939 29793 leader_election.cc:288] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e365d07bbe9444e888b5c38d2d63f1e (127.26.125.2:46731), 0b9a2d66e9214bd79d8a64c8a18d2923 (127.26.125.3:40203)
I20260811 22:25:32.714630 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.710424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:60295 (local address 127.26.125.3:40203)
0811 22:25:32.710555 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.710559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.710887 (+ 328us) server_negotiation.cc:407] Connection header received
0811 22:25:32.711054 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.711058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.711111 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.711182 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:32.711506 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.712024 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.713267 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.713916 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.714050 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.714104 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.714402 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.714529 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:32.714622 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.710465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.125.1:38477 (local address 127.26.125.2:46731)
0811 22:25:32.710606 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.710611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.710625 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:32.710682 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.710684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.710738 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.710829 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:32.711246 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.711855 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.713291 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.713905 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.714050 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.714104 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.714402 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.714481 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:25:32.714658 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.710194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.2:46731 (local address 127.26.125.1:38477)
0811 22:25:32.710347 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.710364 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.710475 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.710838 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.710841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.710896 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:32.711034 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.711042 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.711974 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.711978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.713117 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.713129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.714076 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.714080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.714264 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.714301 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.714380 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.714522 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":164,"spinlock_wait_cycles":2816}
I20260811 22:25:32.714668 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.710269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.3:40203 (local address 127.26.125.1:60295)
0811 22:25:32.710769 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.710782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.710923 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.711239 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.711242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.711264 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:32.711390 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.711397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.712131 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.712134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.713129 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.713141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.714076 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.714080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.714260 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.714312 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.714380 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.714485 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":110,"thread_start_us":154,"threads_started":1}
I20260811 22:25:32.715400 29665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a68f71083804963be884be898806d39" candidate_uuid: "0fa3d5efe1964a8a997e3a5205d78850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" is_pre_election: true
I20260811 22:25:32.715404 29421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a68f71083804963be884be898806d39" candidate_uuid: "0fa3d5efe1964a8a997e3a5205d78850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" is_pre_election: true
I20260811 22:25:32.715713 29421 raft_consensus.cc:2454] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa3d5efe1964a8a997e3a5205d78850 in term 0.
I20260811 22:25:32.715718 29665 raft_consensus.cc:2454] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa3d5efe1964a8a997e3a5205d78850 in term 0.
I20260811 22:25:32.716226 29085 leader_election.cc:302] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b9a2d66e9214bd79d8a64c8a18d2923, 0fa3d5efe1964a8a997e3a5205d78850; no voters:
I20260811 22:25:32.716557 29793 raft_consensus.cc:2781] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:32.716645 29793 raft_consensus.cc:482] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:32.716676 29793 raft_consensus.cc:3037] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.717835 29793 raft_consensus.cc:504] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.718201 29793 leader_election.cc:288] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [CANDIDATE]: Term 1 election: Requested vote from peers 8e365d07bbe9444e888b5c38d2d63f1e (127.26.125.2:46731), 0b9a2d66e9214bd79d8a64c8a18d2923 (127.26.125.3:40203)
I20260811 22:25:32.718590 29665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a68f71083804963be884be898806d39" candidate_uuid: "0fa3d5efe1964a8a997e3a5205d78850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923"
I20260811 22:25:32.718663 29421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a68f71083804963be884be898806d39" candidate_uuid: "0fa3d5efe1964a8a997e3a5205d78850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e365d07bbe9444e888b5c38d2d63f1e"
I20260811 22:25:32.718734 29665 raft_consensus.cc:3037] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.718783 29421 raft_consensus.cc:3037] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.720103 29421 raft_consensus.cc:2454] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa3d5efe1964a8a997e3a5205d78850 in term 1.
I20260811 22:25:32.720103 29665 raft_consensus.cc:2454] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa3d5efe1964a8a997e3a5205d78850 in term 1.
I20260811 22:25:32.720465 29083 leader_election.cc:302] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa3d5efe1964a8a997e3a5205d78850, 8e365d07bbe9444e888b5c38d2d63f1e; no voters:
I20260811 22:25:32.720690 29793 raft_consensus.cc:2781] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:32.720988 29793 raft_consensus.cc:686] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [term 1 LEADER]: Becoming Leader. State: Replica: 0fa3d5efe1964a8a997e3a5205d78850, State: Running, Role: LEADER
I20260811 22:25:32.721371 29793 consensus_queue.cc:237] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } }
I20260811 22:25:32.724687 28956 catalog_manager.cc:5495] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa3d5efe1964a8a997e3a5205d78850 (127.26.125.1). New cstate: current_term: 1 leader_uuid: "0fa3d5efe1964a8a997e3a5205d78850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa3d5efe1964a8a997e3a5205d78850" member_type: VOTER last_known_addr { host: "127.26.125.1" port: 42671 } health_report { overall_health: HEALTHY } } }
W20260811 22:25:32.726528 29268 tablet.cc:2369] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:32.752174 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.748186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58308 (local address 127.26.125.1:42671)
0811 22:25:32.748324 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.748327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.748343 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:32.748454 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.748456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.748515 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.748614 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:32.748920 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.749515 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.750493 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.750718 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.750804 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.751220 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:32.751726 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:32.751836 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.751941 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.752002 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:25:32.758996 29421 raft_consensus.cc:1261] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 1 FOLLOWER]: Refusing update from remote peer 0fa3d5efe1964a8a997e3a5205d78850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:32.758996 29665 raft_consensus.cc:1261] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 1 FOLLOWER]: Refusing update from remote peer 0fa3d5efe1964a8a997e3a5205d78850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:32.759873 29793 consensus_queue.cc:1035] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e365d07bbe9444e888b5c38d2d63f1e" member_type: VOTER last_known_addr { host: "127.26.125.2" port: 46731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:32.760103 29799 consensus_queue.cc:1035] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b9a2d66e9214bd79d8a64c8a18d2923" member_type: VOTER last_known_addr { host: "127.26.125.3" port: 40203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:32.769761 29805 mvcc.cc:205] Tried to move back new op lower bound from 7317451295772262400 to 7317451295629787136. Current Snapshot: MvccSnapshot[applied={T|T < 7317451295772262400}]
I20260811 22:25:32.771375 29804 mvcc.cc:205] Tried to move back new op lower bound from 7317451295772262400 to 7317451295629787136. Current Snapshot: MvccSnapshot[applied={T|T < 7317451295772262400}]
I20260811 22:25:32.779773 28955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:32.779958 28955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:32.782464 28955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb8305a4a2db4657aabbe7a89d404db2: Sending DeleteTablet for 3 replicas of tablet 8a68f71083804963be884be898806d39
I20260811 22:25:32.783376 29384 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a68f71083804963be884be898806d39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:32 UTC) from {username='slave'} at 127.0.0.1:51426
I20260811 22:25:32.783376 29157 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a68f71083804963be884be898806d39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:32 UTC) from {username='slave'} at 127.0.0.1:58296
I20260811 22:25:32.783499 29628 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a68f71083804963be884be898806d39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:32 UTC) from {username='slave'} at 127.0.0.1:53410
I20260811 22:25:32.783954 29814 tablet_replica.cc:330] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: stopping tablet replica
I20260811 22:25:32.784200 29818 tablet_replica.cc:330] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: stopping tablet replica
I20260811 22:25:32.784440 29814 raft_consensus.cc:2229] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:32.784539 29818 raft_consensus.cc:2229] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:32.784794 29816 tablet_replica.cc:330] T 8a68f71083804963be884be898806d39 P 0fa3d5efe1964a8a997e3a5205d78850: stopping tablet replica
I20260811 22:25:32.784821 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 29060
I20260811 22:25:32.784885 29818 raft_consensus.cc:2258] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:32.789579 29814 raft_consensus.cc:2258] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:32.790499 29818 ts_tablet_manager.cc:1905] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:32.791389 29814 ts_tablet_manager.cc:1905] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:32.793630 29818 ts_tablet_manager.cc:1918] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:32.793753 29818 log.cc:1196] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-2/wal/wals/8a68f71083804963be884be898806d39
I20260811 22:25:32.794097 29818 ts_tablet_manager.cc:1939] T 8a68f71083804963be884be898806d39 P 0b9a2d66e9214bd79d8a64c8a18d2923: Deleting consensus metadata
I20260811 22:25:32.794176 29814 ts_tablet_manager.cc:1918] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:32.794279 29814 log.cc:1196] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: Deleting WAL directory at /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486868287078-27124-0/minicluster-data/ts-1/wal/wals/8a68f71083804963be884be898806d39
I20260811 22:25:32.794593 29814 ts_tablet_manager.cc:1939] T 8a68f71083804963be884be898806d39 P 8e365d07bbe9444e888b5c38d2d63f1e: Deleting consensus metadata
W20260811 22:25:32.795644 28935 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.125.1:42671 (error 108)
I20260811 22:25:32.795758 28930 catalog_manager.cc:4841] TS 8e365d07bbe9444e888b5c38d2d63f1e (127.26.125.2:46731): tablet 8a68f71083804963be884be898806d39 (table test-table [id=15d92dfccee44d6b9f03edae2c74492c]) successfully deleted
I20260811 22:25:32.796164 28935 catalog_manager.cc:4841] TS 0b9a2d66e9214bd79d8a64c8a18d2923 (127.26.125.3:40203): tablet 8a68f71083804963be884be898806d39 (table test-table [id=15d92dfccee44d6b9f03edae2c74492c]) successfully deleted
W20260811 22:25:32.796765 29776 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:32.796447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.1:42671 (local address 127.0.0.1:58310)
0811 22:25:32.796614 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.796682 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.1:42671: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 22:25:32.796913 28935 catalog_manager.cc:4584] TS 0fa3d5efe1964a8a997e3a5205d78850 (127.26.125.1:42671): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a68f71083804963be884be898806d39: Network error: Client connection negotiation failed: client connection to 127.26.125.1:42671: connect: Connection reset by peer (error 104)
I20260811 22:25:32.796950 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 29270
I20260811 22:25:32.809917 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 29510
I20260811 22:25:32.826391 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 28898
2026-08-11T22:25:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1122 ms)
[----------] 6 tests from EncryptionPolicyTest (11435 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:32 dist-test-slave-5bn2 krb5kdc[29826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:32 dist-test-slave-5bn2 krb5kdc[29826](info): set up 2 sockets
Aug 11 22:25:32 dist-test-slave-5bn2 krb5kdc[29826](info): commencing operation
krb5kdc: starting...
W20260811 22:25:34.012182 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:34 dist-test-slave-5bn2 krb5kdc[29826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:34.145866 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43959
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:34.267711 29849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:34.268059 29849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:34.268129 29849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:34.272442 29849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:34.272540 29849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:34.272571 29849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:34.272590 29849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:34.272608 29849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:34.278357 29849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43959
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:34.279587 29849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:34.280627 29849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:34.289779 29849 server_base.cc:1034] running on GCE node
W20260811 22:25:34.289847 29863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:34.289757 29858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:34.289847 29856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:34.290723 29849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:34.292022 29849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:34.293205 29849 hybrid_clock.cc:648] HybridClock initialized: now 1786487134293155 us; error 63 us; skew 500 ppm
Aug 11 22:25:34 dist-test-slave-5bn2 krb5kdc[29826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:34.296216 29849 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:34.297226 29849 webserver.cc:460] Webserver started at http://127.0.0.1:42661/ using document root <none> and password file <none>
I20260811 22:25:34.297811 29849 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:34.297875 29849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:34.298091 29849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:34.300343 29849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb"
format_stamp: "Formatted at 2026-08-11 22:25:34 on dist-test-slave-5bn2"
server_key: "a6d7798b6f7f169bd2927d44d2b456a3"
server_key_iv: "a42b77414bc7b617e6becae5c406df1e"
server_key_version: "encryptionkey@0"
I20260811 22:25:34.300966 29849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb"
format_stamp: "Formatted at 2026-08-11 22:25:34 on dist-test-slave-5bn2"
server_key: "a6d7798b6f7f169bd2927d44d2b456a3"
server_key_iv: "a42b77414bc7b617e6becae5c406df1e"
server_key_version: "encryptionkey@0"
I20260811 22:25:34.305173 29849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:34.308010 29873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:34.309437 29849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:34.309576 29849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb"
format_stamp: "Formatted at 2026-08-11 22:25:34 on dist-test-slave-5bn2"
server_key: "a6d7798b6f7f169bd2927d44d2b456a3"
server_key_iv: "a42b77414bc7b617e6becae5c406df1e"
server_key_version: "encryptionkey@0"
I20260811 22:25:34.309687 29849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:34.323621 29849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:34.326567 29849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:34.326759 29849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:34.333854 29849 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43959
I20260811 22:25:34.333873 29959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43959 every 8 connection(s)
I20260811 22:25:34.334964 29849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:34.337913 29960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:34.343680 29960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb: Bootstrap starting.
I20260811 22:25:34.344369 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 29849
I20260811 22:25:34.344511 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:34.344762 27124 external_mini_cluster.cc:1428] Setting key 8cfd53a145553cb1f8b8576ef89e7c89
I20260811 22:25:34.346596 29960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:34.347569 29960 log.cc:824] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:34.349788 29960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb: No bootstrap required, opened a new log
I20260811 22:25:34.353576 29960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43959 } }
I20260811 22:25:34.353832 29960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:34.353902 29960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b53fc686aed4ede81a5f8bfbf19f7fb, State: Initialized, Role: FOLLOWER
Aug 11 22:25:34 dist-test-slave-5bn2 krb5kdc[29826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:25:34.354370 29960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43959 } }
I20260811 22:25:34.354506 29960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:34.354576 29960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:34.354684 29960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:34.355938 29960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43959 } }
I20260811 22:25:34.356339 29960 leader_election.cc:302] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b53fc686aed4ede81a5f8bfbf19f7fb; no voters:
I20260811 22:25:34.356724 29960 leader_election.cc:288] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:34.356885 29969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:34.357113 29969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [term 1 LEADER]: Becoming Leader. State: Replica: 9b53fc686aed4ede81a5f8bfbf19f7fb, State: Running, Role: LEADER
I20260811 22:25:34.357471 29969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43959 } }
I20260811 22:25:34.358213 29960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:34.359313 29971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b53fc686aed4ede81a5f8bfbf19f7fb. Latest consensus state: current_term: 1 leader_uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43959 } } }
I20260811 22:25:34.359462 29971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:34.359826 29970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b53fc686aed4ede81a5f8bfbf19f7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43959 } } }
I20260811 22:25:34.359962 29970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:34.360309 29988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:34.361438 29964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.346141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39385 (local address 127.0.0.1:43959)
0811 22:25:34.346629 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:34.346650 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:34.346698 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:25:34.347611 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:34.347638 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:34.348103 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:34.348442 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:34.349312 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.350400 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.351341 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.351889 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.352063 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.354719 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:34.354727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:34.354746 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:34.354784 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:34.356859 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:34.357564 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:34.357570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:34.357576 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:34.357725 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:34.358161 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:34.358165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:34.358166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:34.358750 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:34.359698 (+ 948us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.359915 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.360194 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":114,"threads_started":1}
I20260811 22:25:34.363937 29988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:34.364545 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:34.370338 29988 catalog_manager.cc:1347] Generated new cluster ID: 70cbefa9166f4b8084d9a07f01ad89c3
I20260811 22:25:34.370414 29988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:34.377333 29964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.366612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48171 (local address 127.0.0.1:43959)
0811 22:25:34.366716 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:34.366734 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:34.367116 (+ 382us) server_negotiation.cc:407] Connection header received
0811 22:25:34.367253 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:34.367256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:34.367305 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:34.367387 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:34.368320 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.368942 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.370200 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.370547 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.370656 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.372686 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:34.372691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:34.372694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:34.372726 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:34.374383 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:34.375151 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:34.375156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:34.375158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:34.375259 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:34.375705 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:34.375710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:34.375712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:34.375973 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:34.376146 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.377040 (+ 894us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.377155 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:25:34.387679 29988 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:34.389052 29988 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:34.397344 29988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b53fc686aed4ede81a5f8bfbf19f7fb: Generated new TSK 0
I20260811 22:25:34.398277 29988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:34.411404 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 29849
W20260811 22:25:34.429606 29821 connection.cc:535] client connection to 127.0.0.1:43959 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43959): Connection reset by peer (error 104)
2026-08-11T22:25:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:34 dist-test-slave-5bn2 krb5kdc[30014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:34 dist-test-slave-5bn2 krb5kdc[30014](info): set up 2 sockets
Aug 11 22:25:34 dist-test-slave-5bn2 krb5kdc[30014](info): commencing operation
krb5kdc: starting...
W20260811 22:25:35.611033 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:35 dist-test-slave-5bn2 krb5kdc[30014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:35.746730 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:35.870415 30040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:35.870743 30040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:35.870816 30040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:35.874969 30040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:35.875063 30040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:35.875097 30040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:35.875123 30040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:35.875155 30040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:35.880906 30040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42961
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:35.882128 30040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:35.883462 30040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:35.894433 30049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:35.894459 30051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:35.894637 30040 server_base.cc:1034] running on GCE node
W20260811 22:25:35.894438 30054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:35.895467 30040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:35.896744 30040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:35.897984 30040 hybrid_clock.cc:648] HybridClock initialized: now 1786487135897952 us; error 43 us; skew 500 ppm
Aug 11 22:25:35 dist-test-slave-5bn2 krb5kdc[30014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487135, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:35.901228 30040 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:35.902473 30040 webserver.cc:460] Webserver started at http://127.0.0.1:42241/ using document root <none> and password file <none>
I20260811 22:25:35.903190 30040 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:35.903264 30040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:35.903491 30040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:35.905877 30040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "2ba113a3ce0847b7bb8d3e6570c762e2"
format_stamp: "Formatted at 2026-08-11 22:25:35 on dist-test-slave-5bn2"
server_key: "4f23a674597d18e3a1c661dd303992d9"
server_key_iv: "627fcbffb8bf23eec1abf21eb8c61229"
server_key_version: "encryptionkey@0"
I20260811 22:25:35.906466 30040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "2ba113a3ce0847b7bb8d3e6570c762e2"
format_stamp: "Formatted at 2026-08-11 22:25:35 on dist-test-slave-5bn2"
server_key: "4f23a674597d18e3a1c661dd303992d9"
server_key_iv: "627fcbffb8bf23eec1abf21eb8c61229"
server_key_version: "encryptionkey@0"
I20260811 22:25:35.910567 30040 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:25:35.913264 30068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:35.914620 30040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:25:35.914764 30040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "2ba113a3ce0847b7bb8d3e6570c762e2"
format_stamp: "Formatted at 2026-08-11 22:25:35 on dist-test-slave-5bn2"
server_key: "4f23a674597d18e3a1c661dd303992d9"
server_key_iv: "627fcbffb8bf23eec1abf21eb8c61229"
server_key_version: "encryptionkey@0"
I20260811 22:25:35.914908 30040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:35.943257 30040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:35.946547 30040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:35.946714 30040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:35.953861 30040 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42961
I20260811 22:25:35.953882 30155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42961 every 8 connection(s)
I20260811 22:25:35.954886 30040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:35.955633 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30040
I20260811 22:25:35.955776 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:35.956113 27124 external_mini_cluster.cc:1428] Setting key 65098c5e735732c98bec4bf71a13b8f3
I20260811 22:25:35.958482 30156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:35.964563 30156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2: Bootstrap starting.
I20260811 22:25:35.965507 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:35.957706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51403 (local address 127.0.0.1:42961)
0811 22:25:35.958182 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:35.958199 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:35.958237 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:25:35.958969 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:35.958992 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:35.959349 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:35.959761 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:35.960198 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:35.961217 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:35.962134 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:35.962637 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:35.962790 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:35.963776 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:35.963781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:35.963795 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:35.964207 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:35.964269 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:35.964293 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:35.964411 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:35.964608 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 22:25:35.964752 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":163,"threads_started":1}
I20260811 22:25:35.967188 30156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:35.968050 30156 log.cc:824] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:35.968657 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:35.970187 30156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2: No bootstrap required, opened a new log
I20260811 22:25:35.973884 30156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba113a3ce0847b7bb8d3e6570c762e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260811 22:25:35.974150 30156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:35.974237 30156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba113a3ce0847b7bb8d3e6570c762e2, State: Initialized, Role: FOLLOWER
I20260811 22:25:35.974409 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:35.970651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59903 (local address 127.0.0.1:42961)
0811 22:25:35.970756 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:35.970761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:35.971012 (+ 251us) server_negotiation.cc:407] Connection header received
0811 22:25:35.971123 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:35.971126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:35.971168 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:35.971232 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:35.971496 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:35.972000 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:35.972817 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:35.973040 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:35.973123 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:35.973911 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:35.973915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:35.973918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:35.973932 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:35.973957 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:35.973968 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:35.974034 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:35.974182 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 22:25:35.974221 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:25:35.974798 30156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba113a3ce0847b7bb8d3e6570c762e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260811 22:25:35.975010 30156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:35.975078 30156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:35.975186 30156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:35.976177 30156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba113a3ce0847b7bb8d3e6570c762e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260811 22:25:35.976521 30156 leader_election.cc:302] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ba113a3ce0847b7bb8d3e6570c762e2; no voters:
I20260811 22:25:35.976838 30156 leader_election.cc:288] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:35.976951 30170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:35.977160 30170 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba113a3ce0847b7bb8d3e6570c762e2, State: Running, Role: LEADER
I20260811 22:25:35.977523 30170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba113a3ce0847b7bb8d3e6570c762e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260811 22:25:35.978097 30156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:35.979200 30173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba113a3ce0847b7bb8d3e6570c762e2. Latest consensus state: current_term: 1 leader_uuid: "2ba113a3ce0847b7bb8d3e6570c762e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba113a3ce0847b7bb8d3e6570c762e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } } }
I20260811 22:25:35.979346 30173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:35.979719 30172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ba113a3ce0847b7bb8d3e6570c762e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba113a3ce0847b7bb8d3e6570c762e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } } }
I20260811 22:25:35.979789 30183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:35.979897 30172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:35.982978 30183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:35.988217 30183 catalog_manager.cc:1347] Generated new cluster ID: 7e1dcbd655fa413e8fd0609e0ea53382
I20260811 22:25:35.988299 30183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:35.997195 30183 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:35.998142 30183 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:36.005956 30183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ba113a3ce0847b7bb8d3e6570c762e2: Generated new TSK 0
I20260811 22:25:36.006776 30183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:36.015776 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30040
W20260811 22:25:36.026733 30011 connection.cc:535] client connection to 127.0.0.1:42961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42961): Connection reset by peer (error 104)
2026-08-11T22:25:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:36 dist-test-slave-5bn2 krb5kdc[30216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:36 dist-test-slave-5bn2 krb5kdc[30216](info): set up 2 sockets
Aug 11 22:25:36 dist-test-slave-5bn2 krb5kdc[30216](info): commencing operation
krb5kdc: starting...
W20260811 22:25:37.185837 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:37 dist-test-slave-5bn2 krb5kdc[30216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:37.318011 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:37.433498 30237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:37.433813 30237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:37.433877 30237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:37.437979 30237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:37.438055 30237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:37.438082 30237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:37.438102 30237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:37.438123 30237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:37.443583 30237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:37.444793 30237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:37.445876 30237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:37.454726 30247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:37.454774 30237 server_base.cc:1034] running on GCE node
W20260811 22:25:37.454794 30255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:37.454726 30253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:37.455613 30237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:37.456874 30237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:37.458040 30237 hybrid_clock.cc:648] HybridClock initialized: now 1786487137458024 us; error 31 us; skew 500 ppm
Aug 11 22:25:37 dist-test-slave-5bn2 krb5kdc[30216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487137, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:37.465241 30237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:37.466360 30237 webserver.cc:460] Webserver started at http://127.0.0.1:34545/ using document root <none> and password file <none>
I20260811 22:25:37.467028 30237 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:37.467111 30237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:37.467343 30237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:37.469525 30237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "1c621320bffc42939f457d2c2d64f382"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-5bn2"
server_key: "e97c5259ee80e0654e072d1cfb1055de"
server_key_iv: "8540ab75bcb399866362fe65b07e5e15"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.470115 30237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "1c621320bffc42939f457d2c2d64f382"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-5bn2"
server_key: "e97c5259ee80e0654e072d1cfb1055de"
server_key_iv: "8540ab75bcb399866362fe65b07e5e15"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.474048 30237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 22:25:37.476715 30264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:37.478106 30237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:37.478250 30237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "1c621320bffc42939f457d2c2d64f382"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-5bn2"
server_key: "e97c5259ee80e0654e072d1cfb1055de"
server_key_iv: "8540ab75bcb399866362fe65b07e5e15"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.478368 30237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:37.483305 30237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:37.486524 30237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:37.486698 30237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:37.494024 30237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46459
I20260811 22:25:37.494037 30353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46459 every 8 connection(s)
I20260811 22:25:37.495226 30237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:37.495458 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30237
I20260811 22:25:37.495610 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:37.495961 27124 external_mini_cluster.cc:1428] Setting key c3567873c4aaca4f642d0736d13a7ff4
I20260811 22:25:37.499101 30354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:37.501943 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:37.497370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49425 (local address 127.0.0.1:46459)
0811 22:25:37.497819 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:37.497839 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:37.497883 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:25:37.498756 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:37.498783 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:37.499222 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:37.499653 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:37.500430 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:37.500436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:37.500452 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:37.500464 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:37.500540 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:37.500568 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:37.500719 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:37.500792 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 22:25:37.500956 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":128,"threads_started":1}
I20260811 22:25:37.505237 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:37.506186 30354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382: Bootstrap starting.
I20260811 22:25:37.508764 30354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:37.509106 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:37.507364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36491 (local address 127.0.0.1:46459)
0811 22:25:37.507513 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:37.507517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:37.507655 (+ 138us) server_negotiation.cc:407] Connection header received
0811 22:25:37.507783 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:37.507786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:37.507823 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:37.507887 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:37.508729 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:37.508733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:37.508736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:37.508748 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:37.508777 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:37.508788 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:37.508861 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:37.508943 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 22:25:37.508985 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:25:37.509644 30354 log.cc:824] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:37.511780 30354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382: No bootstrap required, opened a new log
I20260811 22:25:37.515556 30354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c621320bffc42939f457d2c2d64f382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260811 22:25:37.515821 30354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:37.515861 30354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c621320bffc42939f457d2c2d64f382, State: Initialized, Role: FOLLOWER
I20260811 22:25:37.516338 30354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c621320bffc42939f457d2c2d64f382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260811 22:25:37.516486 30354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:37.516558 30354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:37.516672 30354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:37.517652 30354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c621320bffc42939f457d2c2d64f382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260811 22:25:37.518018 30354 leader_election.cc:302] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c621320bffc42939f457d2c2d64f382; no voters:
I20260811 22:25:37.518355 30354 leader_election.cc:288] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:37.518481 30371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:37.518728 30371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [term 1 LEADER]: Becoming Leader. State: Replica: 1c621320bffc42939f457d2c2d64f382, State: Running, Role: LEADER
I20260811 22:25:37.519115 30371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c621320bffc42939f457d2c2d64f382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260811 22:25:37.519626 30354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:37.520841 30373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c621320bffc42939f457d2c2d64f382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c621320bffc42939f457d2c2d64f382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } } }
I20260811 22:25:37.520990 30373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:37.521289 30374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c621320bffc42939f457d2c2d64f382. Latest consensus state: current_term: 1 leader_uuid: "1c621320bffc42939f457d2c2d64f382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c621320bffc42939f457d2c2d64f382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } } }
I20260811 22:25:37.521384 30374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:37.521433 30385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:37.524718 30385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:37.530709 30385 catalog_manager.cc:1347] Generated new cluster ID: 0922546c16ba46558fd217ae3d3bd618
I20260811 22:25:37.530802 30385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:37.551671 30385 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:37.552117 30385 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:37.558109 30385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c621320bffc42939f457d2c2d64f382: Generated new TSK 0
I20260811 22:25:37.559043 30385 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:37.570042 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30237
2026-08-11T22:25:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:37 dist-test-slave-5bn2 krb5kdc[30403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:37 dist-test-slave-5bn2 krb5kdc[30403](info): set up 2 sockets
Aug 11 22:25:37 dist-test-slave-5bn2 krb5kdc[30403](info): commencing operation
krb5kdc: starting...
W20260811 22:25:38.760130 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:38 dist-test-slave-5bn2 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:38.902364 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:39.021806 30430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:39.022136 30430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:39.022198 30430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:39.026468 30430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:39.026572 30430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:39.026599 30430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:39.026619 30430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:39.026638 30430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:39.032456 30430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:39.033696 30430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:39.034776 30430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:39.043646 30435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:39.043646 30438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:39.043710 30430 server_base.cc:1034] running on GCE node
W20260811 22:25:39.043650 30436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:39.044553 30430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:39.045814 30430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:39.046990 30430 hybrid_clock.cc:648] HybridClock initialized: now 1786487139046966 us; error 42 us; skew 500 ppm
Aug 11 22:25:39 dist-test-slave-5bn2 krb5kdc[30403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487139, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:39.050110 30430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:39.051175 30430 webserver.cc:460] Webserver started at http://127.0.0.1:46487/ using document root <none> and password file <none>
I20260811 22:25:39.051807 30430 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:39.051883 30430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:39.052114 30430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:39.054316 30430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "dc4a7cc9284f4bc0b46816f1877a647c"
format_stamp: "Formatted at 2026-08-11 22:25:39 on dist-test-slave-5bn2"
server_key: "428fffd1d7dcb24d56ad19ef15451aca"
server_key_iv: "710cf4840a9304ed777449e72fe8332e"
server_key_version: "encryptionkey@0"
I20260811 22:25:39.054979 30430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "dc4a7cc9284f4bc0b46816f1877a647c"
format_stamp: "Formatted at 2026-08-11 22:25:39 on dist-test-slave-5bn2"
server_key: "428fffd1d7dcb24d56ad19ef15451aca"
server_key_iv: "710cf4840a9304ed777449e72fe8332e"
server_key_version: "encryptionkey@0"
I20260811 22:25:39.059403 30430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:25:39.062258 30452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:39.063741 30430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:39.063881 30430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "dc4a7cc9284f4bc0b46816f1877a647c"
format_stamp: "Formatted at 2026-08-11 22:25:39 on dist-test-slave-5bn2"
server_key: "428fffd1d7dcb24d56ad19ef15451aca"
server_key_iv: "710cf4840a9304ed777449e72fe8332e"
server_key_version: "encryptionkey@0"
I20260811 22:25:39.063995 30430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:39.081655 30430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:39.085368 30430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:39.085583 30430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:39.093328 30430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36329
I20260811 22:25:39.093393 30549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36329 every 8 connection(s)
I20260811 22:25:39.094403 30430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:39.097548 30551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:39.100737 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30430
I20260811 22:25:39.100909 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:39.101261 27124 external_mini_cluster.cc:1428] Setting key 68a5d5fbfdf698677c8733c53f6f30e0
I20260811 22:25:39.104105 30551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c: Bootstrap starting.
I20260811 22:25:39.106971 30551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:39.108036 30551 log.cc:824] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:39.110782 30551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c: No bootstrap required, opened a new log
Aug 11 22:25:39 dist-test-slave-5bn2 krb5kdc[30403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:25:39.114467 30551 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a7cc9284f4bc0b46816f1877a647c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } }
I20260811 22:25:39.114809 30551 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:39.114900 30551 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc4a7cc9284f4bc0b46816f1877a647c, State: Initialized, Role: FOLLOWER
I20260811 22:25:39.115433 30551 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a7cc9284f4bc0b46816f1877a647c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } }
I20260811 22:25:39.115617 30551 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:39.115689 30551 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:39.115794 30551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:39.116998 30551 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a7cc9284f4bc0b46816f1877a647c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } }
I20260811 22:25:39.117355 30551 leader_election.cc:302] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc4a7cc9284f4bc0b46816f1877a647c; no voters:
I20260811 22:25:39.117703 30551 leader_election.cc:288] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:39.117830 30559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:39.117971 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:39.102889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41497 (local address 127.0.0.1:36329)
0811 22:25:39.103364 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:39.103381 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:39.103420 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:25:39.104105 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:39.104125 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:39.104568 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:39.104898 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:39.105911 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:39.107109 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:39.108380 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:39.109016 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:39.109152 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:39.112149 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:39.112155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:39.112172 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:39.112206 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:39.114496 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:39.115073 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:39.115079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:39.115085 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:39.115244 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:39.115573 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:39.115577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:39.115578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:39.116174 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:39.116368 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:39.116644 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:25:39.116975 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":140,"threads_started":1}
I20260811 22:25:39.118064 30559 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [term 1 LEADER]: Becoming Leader. State: Replica: dc4a7cc9284f4bc0b46816f1877a647c, State: Running, Role: LEADER
I20260811 22:25:39.118582 30559 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a7cc9284f4bc0b46816f1877a647c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } }
I20260811 22:25:39.119076 30551 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:39.120960 30562 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc4a7cc9284f4bc0b46816f1877a647c. Latest consensus state: current_term: 1 leader_uuid: "dc4a7cc9284f4bc0b46816f1877a647c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a7cc9284f4bc0b46816f1877a647c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } } }
I20260811 22:25:39.121367 30562 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:39.121361 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:39.121258 30560 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc4a7cc9284f4bc0b46816f1877a647c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a7cc9284f4bc0b46816f1877a647c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } } }
I20260811 22:25:39.122066 30560 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c [sys.catalog]: This master's current role is: LEADER
W20260811 22:25:39.124584 30576 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:25:39.124658 30576 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:25:39.124740 30580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:39.125681 30580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:39.132566 30580 catalog_manager.cc:1347] Generated new cluster ID: 6d500f2b4a5f4a2bb5364f9d65f72213
I20260811 22:25:39.132746 30580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:39.133314 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:39.123630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51579 (local address 127.0.0.1:36329)
0811 22:25:39.123749 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:39.123753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:39.125067 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 22:25:39.125194 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:39.125198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:39.125254 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:39.125345 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:39.126217 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:39.127090 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:39.128058 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:39.128281 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:39.128376 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:39.129718 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:39.129721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:39.129724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:39.129751 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:39.131464 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:39.131993 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:39.131996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:39.131998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:39.132092 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:39.132420 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:39.132423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:39.132425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:39.132652 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:39.132806 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:39.133048 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:25:39.133165 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:25:39.143546 30580 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:39.144623 30580 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:39.150537 30580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc4a7cc9284f4bc0b46816f1877a647c: Generated new TSK 0
I20260811 22:25:39.151422 30580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:39.157765 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30430
2026-08-11T22:25:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:39 dist-test-slave-5bn2 krb5kdc[30596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:39 dist-test-slave-5bn2 krb5kdc[30596](info): set up 2 sockets
Aug 11 22:25:39 dist-test-slave-5bn2 krb5kdc[30596](info): commencing operation
krb5kdc: starting...
W20260811 22:25:40.349294 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:40 dist-test-slave-5bn2 krb5kdc[30596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:40.484256 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:40.604521 30618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:40.604875 30618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:40.604939 30618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:40.609771 30618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:40.609871 30618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:40.609901 30618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:40.609923 30618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:40.609943 30618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:40.615820 30618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:40.617005 30618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:40.618125 30618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:40.626744 30630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:40.626936 30618 server_base.cc:1034] running on GCE node
W20260811 22:25:40.626765 30636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:40.626865 30627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:40.627676 30618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:40.628881 30618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:40.630074 30618 hybrid_clock.cc:648] HybridClock initialized: now 1786487140630056 us; error 34 us; skew 500 ppm
Aug 11 22:25:40 dist-test-slave-5bn2 krb5kdc[30596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487140, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:40.632965 30618 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:40.633904 30618 webserver.cc:460] Webserver started at http://127.0.0.1:37723/ using document root <none> and password file <none>
I20260811 22:25:40.634457 30618 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:40.634521 30618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:40.634717 30618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:40.636896 30618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "53aff30750bf4520bbcda619b571edb8"
format_stamp: "Formatted at 2026-08-11 22:25:40 on dist-test-slave-5bn2"
server_key: "d532a6efb83d5f4e60c57ab3bf5eecfe"
server_key_iv: "f2c551dff945192a02d08487c7f870ba"
server_key_version: "encryptionkey@0"
I20260811 22:25:40.637461 30618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "53aff30750bf4520bbcda619b571edb8"
format_stamp: "Formatted at 2026-08-11 22:25:40 on dist-test-slave-5bn2"
server_key: "d532a6efb83d5f4e60c57ab3bf5eecfe"
server_key_iv: "f2c551dff945192a02d08487c7f870ba"
server_key_version: "encryptionkey@0"
I20260811 22:25:40.641438 30618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 22:25:40.643950 30649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:40.645223 30618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:25:40.645337 30618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "53aff30750bf4520bbcda619b571edb8"
format_stamp: "Formatted at 2026-08-11 22:25:40 on dist-test-slave-5bn2"
server_key: "d532a6efb83d5f4e60c57ab3bf5eecfe"
server_key_iv: "f2c551dff945192a02d08487c7f870ba"
server_key_version: "encryptionkey@0"
I20260811 22:25:40.645431 30618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:40.662902 30618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:40.666216 30618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:40.666394 30618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:40.673661 30618 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43461
I20260811 22:25:40.673683 30744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43461 every 8 connection(s)
I20260811 22:25:40.674749 30618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:40.677755 30746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:40.682950 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30618
I20260811 22:25:40.683126 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:40.683416 27124 external_mini_cluster.cc:1428] Setting key ff188cc5921775644aef50999574c6d4
I20260811 22:25:40.683918 30746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8: Bootstrap starting.
I20260811 22:25:40.686458 30746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:40.687393 30746 log.cc:824] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:40.689707 30746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8: No bootstrap required, opened a new log
I20260811 22:25:40.692844 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:40.684889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54067 (local address 127.0.0.1:43461)
0811 22:25:40.685309 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:40.685326 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:40.685364 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:25:40.686047 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:40.686066 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:40.686461 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:40.686820 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:40.687237 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:40.688268 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:40.689496 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:40.690118 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:40.690258 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:40.691129 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:40.691134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:40.691149 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:40.691544 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:40.691601 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:40.691624 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:40.691699 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:40.691843 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 22:25:40.692066 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":111,"threads_started":1}
I20260811 22:25:40.693740 30746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53aff30750bf4520bbcda619b571edb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43461 } }
I20260811 22:25:40.694008 30746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:40.694072 30746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53aff30750bf4520bbcda619b571edb8, State: Initialized, Role: FOLLOWER
I20260811 22:25:40.694568 30746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53aff30750bf4520bbcda619b571edb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43461 } }
I20260811 22:25:40.694741 30746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:40.694811 30746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:40.694942 30746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:40.695996 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:40.696141 30746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53aff30750bf4520bbcda619b571edb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43461 } }
I20260811 22:25:40.696532 30746 leader_election.cc:302] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53aff30750bf4520bbcda619b571edb8; no voters:
I20260811 22:25:40.696906 30746 leader_election.cc:288] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:40.696990 30759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:40.697196 30759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [term 1 LEADER]: Becoming Leader. State: Replica: 53aff30750bf4520bbcda619b571edb8, State: Running, Role: LEADER
I20260811 22:25:40.697587 30759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53aff30750bf4520bbcda619b571edb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43461 } }
I20260811 22:25:40.698423 30746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:40.698825 30763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53aff30750bf4520bbcda619b571edb8. Latest consensus state: current_term: 1 leader_uuid: "53aff30750bf4520bbcda619b571edb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53aff30750bf4520bbcda619b571edb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43461 } } }
I20260811 22:25:40.699018 30763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:40.699270 30760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53aff30750bf4520bbcda619b571edb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53aff30750bf4520bbcda619b571edb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43461 } } }
I20260811 22:25:40.699383 30760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:40.699563 30773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:40.703035 30773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:40.703483 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:40.698386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52043 (local address 127.0.0.1:43461)
0811 22:25:40.698495 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:40.698500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:40.698959 (+ 459us) server_negotiation.cc:407] Connection header received
0811 22:25:40.699154 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:40.699158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:40.699208 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:40.699261 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:40.699704 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:40.700465 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:40.701660 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:40.701979 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:40.702095 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:40.702992 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:40.702997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:40.703000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:40.703013 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:40.703053 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:40.703069 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:40.703115 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:40.703287 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 22:25:40.703349 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:25:40.708971 30773 catalog_manager.cc:1347] Generated new cluster ID: becfe75c4a184dc69569d7aa365694e6
I20260811 22:25:40.709055 30773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:40.719910 30773 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:40.720981 30773 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:40.728562 30773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53aff30750bf4520bbcda619b571edb8: Generated new TSK 0
I20260811 22:25:40.729336 30773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:40.740011 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30618
2026-08-11T22:25:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1577 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:40 dist-test-slave-5bn2 krb5kdc[30803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:40 dist-test-slave-5bn2 krb5kdc[30803](info): set up 2 sockets
Aug 11 22:25:40 dist-test-slave-5bn2 krb5kdc[30803](info): commencing operation
krb5kdc: starting...
W20260811 22:25:41.925976 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:42 dist-test-slave-5bn2 krb5kdc[30803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:42.066033 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:42.184856 30828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:42.185210 30828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:42.185272 30828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:42.189469 30828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:42.189565 30828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:42.189594 30828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:42.189612 30828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:42.189632 30828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:42.195668 30828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:42.196956 30828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:42.198077 30828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:42.207239 30838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:42.207284 30828 server_base.cc:1034] running on GCE node
W20260811 22:25:42.207208 30845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:42.207208 30841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:42.208150 30828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:42.209368 30828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:42.210566 30828 hybrid_clock.cc:648] HybridClock initialized: now 1786487142210535 us; error 44 us; skew 500 ppm
Aug 11 22:25:42 dist-test-slave-5bn2 krb5kdc[30803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487142, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:42.213726 30828 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:42.214715 30828 webserver.cc:460] Webserver started at http://127.0.0.1:33229/ using document root <none> and password file <none>
I20260811 22:25:42.215343 30828 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:42.215422 30828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:42.215605 30828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:42.217746 30828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "1ef83e8a495e456aac417c123c60cec1"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-5bn2"
server_key: "3c42b5629b5cfd99e2f65e900801030b"
server_key_iv: "378f2ba1ed369ae2f93b1e5d9426258a"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.218276 30828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "1ef83e8a495e456aac417c123c60cec1"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-5bn2"
server_key: "3c42b5629b5cfd99e2f65e900801030b"
server_key_iv: "378f2ba1ed369ae2f93b1e5d9426258a"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.222307 30828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260811 22:25:42.225080 30855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.226482 30828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:25:42.226598 30828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "1ef83e8a495e456aac417c123c60cec1"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-5bn2"
server_key: "3c42b5629b5cfd99e2f65e900801030b"
server_key_iv: "378f2ba1ed369ae2f93b1e5d9426258a"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.226694 30828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:42.231496 30828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:42.235005 30828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:42.235163 30828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:42.241984 30828 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42553
I20260811 22:25:42.242015 30955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42553 every 8 connection(s)
I20260811 22:25:42.243001 30828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:42.244091 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 30828
I20260811 22:25:42.244225 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:42.244532 27124 external_mini_cluster.cc:1428] Setting key 16689f48b176d7b3c8dc74ba222b2921
I20260811 22:25:42.246313 30957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:42.250672 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.246108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38291 (local address 127.0.0.1:42553)
0811 22:25:42.246551 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:42.246571 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:42.246615 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:25:42.247508 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:42.247533 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:42.247918 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:42.248301 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:42.249151 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:42.249157 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:42.249174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:42.249188 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:42.249254 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:42.249283 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:42.249443 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:42.249528 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 22:25:42.249692 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260811 22:25:42.253232 30957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1: Bootstrap starting.
I20260811 22:25:42.253546 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:42.256184 30957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:42.257140 30957 log.cc:824] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:42.257897 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.255986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51719 (local address 127.0.0.1:42553)
0811 22:25:42.256119 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:42.256123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:42.256361 (+ 238us) server_negotiation.cc:407] Connection header received
0811 22:25:42.256474 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:42.256477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:42.256519 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:42.256608 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:42.257346 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:42.257349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:42.257352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:42.257363 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:42.257395 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:42.257408 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:42.257522 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:42.257728 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 22:25:42.257771 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:25:42.259574 30957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1: No bootstrap required, opened a new log
I20260811 22:25:42.263396 30957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef83e8a495e456aac417c123c60cec1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } }
I20260811 22:25:42.263692 30957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:42.263739 30957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef83e8a495e456aac417c123c60cec1, State: Initialized, Role: FOLLOWER
I20260811 22:25:42.264161 30957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef83e8a495e456aac417c123c60cec1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } }
I20260811 22:25:42.264276 30957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:42.264323 30957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:42.264400 30957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:42.265399 30957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef83e8a495e456aac417c123c60cec1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } }
I20260811 22:25:42.265755 30957 leader_election.cc:302] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef83e8a495e456aac417c123c60cec1; no voters:
I20260811 22:25:42.266076 30957 leader_election.cc:288] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:42.266201 30972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:42.266496 30972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef83e8a495e456aac417c123c60cec1, State: Running, Role: LEADER
I20260811 22:25:42.266813 30972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef83e8a495e456aac417c123c60cec1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } }
I20260811 22:25:42.267414 30957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:42.268692 30973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ef83e8a495e456aac417c123c60cec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef83e8a495e456aac417c123c60cec1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } } }
I20260811 22:25:42.268836 30973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:42.269034 30975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ef83e8a495e456aac417c123c60cec1. Latest consensus state: current_term: 1 leader_uuid: "1ef83e8a495e456aac417c123c60cec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef83e8a495e456aac417c123c60cec1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } } }
I20260811 22:25:42.269194 30975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:42.269376 30984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:42.272876 30984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:42.279160 30984 catalog_manager.cc:1347] Generated new cluster ID: e510ec09647c418eb73c814405393ba3
I20260811 22:25:42.279246 30984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:42.291287 30984 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:42.291661 30984 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:42.297042 30984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ef83e8a495e456aac417c123c60cec1: Generated new TSK 0
I20260811 22:25:42.297848 30984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:42.304364 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 30828
2026-08-11T22:25:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1568 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:42 dist-test-slave-5bn2 krb5kdc[31010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:42 dist-test-slave-5bn2 krb5kdc[31010](info): set up 2 sockets
Aug 11 22:25:42 dist-test-slave-5bn2 krb5kdc[31010](info): commencing operation
krb5kdc: starting...
W20260811 22:25:43.491941 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:43 dist-test-slave-5bn2 krb5kdc[31010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:43.628075 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39381
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:43.749397 31038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:43.749719 31038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:43.749763 31038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:43.753916 31038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:43.753993 31038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:43.754012 31038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:43.754027 31038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:43.754041 31038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:43.759581 31038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:43.760715 31038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:43.761804 31038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:43.770776 31049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:43.770838 31038 server_base.cc:1034] running on GCE node
W20260811 22:25:43.770849 31047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:43.770776 31051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:43.771658 31038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:43.772864 31038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:43.774044 31038 hybrid_clock.cc:648] HybridClock initialized: now 1786487143774010 us; error 44 us; skew 500 ppm
Aug 11 22:25:43 dist-test-slave-5bn2 krb5kdc[31010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487143, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:43.780191 31038 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:43.781210 31038 webserver.cc:460] Webserver started at http://127.0.0.1:34831/ using document root <none> and password file <none>
I20260811 22:25:43.781813 31038 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:43.781888 31038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:43.782114 31038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:43.784335 31038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "5830f008b4a749b185e5bd124e445682"
format_stamp: "Formatted at 2026-08-11 22:25:43 on dist-test-slave-5bn2"
server_key: "ce166bba258f73e7a306fc8cf5d0617e"
server_key_iv: "493802c01c47b055ec772ce1c2e16b36"
server_key_version: "encryptionkey@0"
I20260811 22:25:43.784917 31038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "5830f008b4a749b185e5bd124e445682"
format_stamp: "Formatted at 2026-08-11 22:25:43 on dist-test-slave-5bn2"
server_key: "ce166bba258f73e7a306fc8cf5d0617e"
server_key_iv: "493802c01c47b055ec772ce1c2e16b36"
server_key_version: "encryptionkey@0"
I20260811 22:25:43.789132 31038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:43.792052 31065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:43.793443 31038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:25:43.793607 31038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "5830f008b4a749b185e5bd124e445682"
format_stamp: "Formatted at 2026-08-11 22:25:43 on dist-test-slave-5bn2"
server_key: "ce166bba258f73e7a306fc8cf5d0617e"
server_key_iv: "493802c01c47b055ec772ce1c2e16b36"
server_key_version: "encryptionkey@0"
I20260811 22:25:43.793723 31038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:43.814411 31038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:43.817968 31038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:43.818169 31038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:43.825320 31038 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39381
I20260811 22:25:43.825361 31158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39381 every 8 connection(s)
I20260811 22:25:43.826401 31038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:43.829528 31159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:43.835603 31159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682: Bootstrap starting.
I20260811 22:25:43.836530 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31038
I20260811 22:25:43.836684 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:43.836980 27124 external_mini_cluster.cc:1428] Setting key e43c41900fa559cd892cd6a6dffa4b54
I20260811 22:25:43.838753 31159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:43.839535 31159 log.cc:824] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:43.841524 31159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682: No bootstrap required, opened a new log
I20260811 22:25:43.845036 31159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5830f008b4a749b185e5bd124e445682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39381 } }
I20260811 22:25:43.845301 31159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:43.845362 31159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5830f008b4a749b185e5bd124e445682, State: Initialized, Role: FOLLOWER
Aug 11 22:25:43 dist-test-slave-5bn2 krb5kdc[31010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:25:43.845914 31159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5830f008b4a749b185e5bd124e445682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39381 } }
I20260811 22:25:43.846067 31159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:43.846136 31159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:43.846237 31159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:43.847281 31159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5830f008b4a749b185e5bd124e445682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39381 } }
I20260811 22:25:43.847663 31159 leader_election.cc:302] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5830f008b4a749b185e5bd124e445682; no voters:
I20260811 22:25:43.847996 31159 leader_election.cc:288] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:43.848089 31170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:43.848260 31170 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [term 1 LEADER]: Becoming Leader. State: Replica: 5830f008b4a749b185e5bd124e445682, State: Running, Role: LEADER
I20260811 22:25:43.848627 31170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5830f008b4a749b185e5bd124e445682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39381 } }
I20260811 22:25:43.849416 31159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:43.850014 31170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5830f008b4a749b185e5bd124e445682. Latest consensus state: current_term: 1 leader_uuid: "5830f008b4a749b185e5bd124e445682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5830f008b4a749b185e5bd124e445682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39381 } } }
I20260811 22:25:43.850212 31170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:43.850461 31172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5830f008b4a749b185e5bd124e445682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5830f008b4a749b185e5bd124e445682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39381 } } }
I20260811 22:25:43.850575 31172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:43.850766 31183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:43.852411 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:43.838221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:46283 (local address 127.0.0.1:39381)
0811 22:25:43.838731 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:43.838747 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:43.838785 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:25:43.839551 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:43.839573 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:43.839975 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:43.840300 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:43.841139 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:43.842170 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:43.843097 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:43.843605 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:43.843757 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:43.846520 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:43.846525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:43.846540 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:43.846570 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:43.848371 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:43.848977 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:43.848982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:43.848987 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:43.849098 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:43.849424 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:43.849436 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:43.849438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:43.849944 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:43.850968 (+ 1024us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:43.851153 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 22:25:43.851390 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":144,"threads_started":1}
I20260811 22:25:43.854228 31183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:43.856189 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:43.860927 31183 catalog_manager.cc:1347] Generated new cluster ID: 11b5ed81e9354c18b76d55aa22975567
I20260811 22:25:43.861011 31183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:43.867880 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:43.858323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:37235 (local address 127.0.0.1:39381)
0811 22:25:43.858435 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:43.858439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:43.858685 (+ 246us) server_negotiation.cc:407] Connection header received
0811 22:25:43.858832 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:43.858835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:43.858921 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:43.859013 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:43.860185 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:43.861017 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:43.862108 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:43.862510 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:43.862617 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:43.864294 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:43.864298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:43.864301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:43.864330 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:43.866033 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:43.866567 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:43.866571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:43.866573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:43.866668 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:43.866977 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:43.866981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:43.866983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:43.867207 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:43.867361 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:43.867593 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 22:25:43.867703 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:25:43.881317 31183 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:43.882947 31183 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:43.894500 31183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5830f008b4a749b185e5bd124e445682: Generated new TSK 0
I20260811 22:25:43.895469 31183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:43.906499 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31038
W20260811 22:25:43.931819 31006 connection.cc:535] client connection to 127.0.0.1:39381 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39381): Connection reset by peer (error 104)
2026-08-11T22:25:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:43 dist-test-slave-5bn2 krb5kdc[31216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:43 dist-test-slave-5bn2 krb5kdc[31216](info): set up 2 sockets
Aug 11 22:25:43 dist-test-slave-5bn2 krb5kdc[31216](info): commencing operation
krb5kdc: starting...
W20260811 22:25:45.138634 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:45 dist-test-slave-5bn2 krb5kdc[31216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:45.268451 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43291
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:45.386837 31237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:45.387197 31237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:45.387261 31237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:45.391502 31237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:45.391599 31237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:45.391628 31237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:45.391649 31237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:45.391669 31237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:45.397409 31237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:45.398661 31237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:45.399816 31237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:45.408939 31243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:45.409121 31237 server_base.cc:1034] running on GCE node
W20260811 22:25:45.408949 31250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:45.408969 31244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:45.409833 31237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:45.411091 31237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:45.412271 31237 hybrid_clock.cc:648] HybridClock initialized: now 1786487145412240 us; error 46 us; skew 500 ppm
Aug 11 22:25:45 dist-test-slave-5bn2 krb5kdc[31216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:45.415349 31237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:45.416329 31237 webserver.cc:460] Webserver started at http://127.0.0.1:38603/ using document root <none> and password file <none>
I20260811 22:25:45.416939 31237 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:45.417013 31237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:45.417225 31237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:45.419358 31237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "8393da843d7148aaabe74dc2810a1bf6"
format_stamp: "Formatted at 2026-08-11 22:25:45 on dist-test-slave-5bn2"
server_key: "089a23a7bb80a314237af187294d7ea0"
server_key_iv: "206ada5b405da1462d94a34b40be163a"
server_key_version: "encryptionkey@0"
I20260811 22:25:45.419911 31237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "8393da843d7148aaabe74dc2810a1bf6"
format_stamp: "Formatted at 2026-08-11 22:25:45 on dist-test-slave-5bn2"
server_key: "089a23a7bb80a314237af187294d7ea0"
server_key_iv: "206ada5b405da1462d94a34b40be163a"
server_key_version: "encryptionkey@0"
I20260811 22:25:45.423702 31237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:25:45.426182 31259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:45.427608 31237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:45.427752 31237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "8393da843d7148aaabe74dc2810a1bf6"
format_stamp: "Formatted at 2026-08-11 22:25:45 on dist-test-slave-5bn2"
server_key: "089a23a7bb80a314237af187294d7ea0"
server_key_iv: "206ada5b405da1462d94a34b40be163a"
server_key_version: "encryptionkey@0"
I20260811 22:25:45.427874 31237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:45.450533 31237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:45.453810 31237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:45.454002 31237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:45.460997 31237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43291
I20260811 22:25:45.461056 31354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43291 every 8 connection(s)
I20260811 22:25:45.461982 31237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:45.464881 31355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:45.466727 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31237
I20260811 22:25:45.466882 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:45.467159 27124 external_mini_cluster.cc:1428] Setting key 22b0098d91aa893e0950dbad0367548a
I20260811 22:25:45.472330 31355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6: Bootstrap starting.
I20260811 22:25:45.475296 31355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:45.476141 31355 log.cc:824] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:45.478459 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:45.468712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:54753 (local address 127.0.0.1:43291)
0811 22:25:45.469207 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:45.469229 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:45.469275 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:25:45.470107 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:45.470136 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:45.470657 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:45.471127 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:45.471600 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:45.473076 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:45.474115 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:45.474821 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:45.475023 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:45.476040 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:45.476047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:45.476066 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:45.476618 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:45.476683 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:45.476709 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:45.476818 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:45.477053 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:25:45.477482 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":137,"threads_started":1}
I20260811 22:25:45.478706 31355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6: No bootstrap required, opened a new log
I20260811 22:25:45.481783 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:45.482787 31355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8393da843d7148aaabe74dc2810a1bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } }
I20260811 22:25:45.483135 31355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:45.483246 31355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8393da843d7148aaabe74dc2810a1bf6, State: Initialized, Role: FOLLOWER
I20260811 22:25:45.483698 31355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8393da843d7148aaabe74dc2810a1bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } }
I20260811 22:25:45.483908 31355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:45.483987 31355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:45.484134 31355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:45.485303 31355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8393da843d7148aaabe74dc2810a1bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } }
I20260811 22:25:45.485775 31355 leader_election.cc:302] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8393da843d7148aaabe74dc2810a1bf6; no voters:
I20260811 22:25:45.486155 31355 leader_election.cc:288] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:45.486241 31373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:45.486455 31373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [term 1 LEADER]: Becoming Leader. State: Replica: 8393da843d7148aaabe74dc2810a1bf6, State: Running, Role: LEADER
I20260811 22:25:45.486884 31373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8393da843d7148aaabe74dc2810a1bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } }
I20260811 22:25:45.487715 31355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:45.488710 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:45.483877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:49451 (local address 127.0.0.1:43291)
0811 22:25:45.484014 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:45.484018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:45.484276 (+ 258us) server_negotiation.cc:407] Connection header received
0811 22:25:45.484435 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:45.484437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:45.484484 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:45.484551 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:45.484838 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:45.485363 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:45.486589 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:45.486966 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:45.487051 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:45.488121 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:45.488124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:45.488127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:45.488141 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:45.488176 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:45.488189 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:45.488279 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:45.488511 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 22:25:45.488556 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:25:45.488761 31375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8393da843d7148aaabe74dc2810a1bf6. Latest consensus state: current_term: 1 leader_uuid: "8393da843d7148aaabe74dc2810a1bf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8393da843d7148aaabe74dc2810a1bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } } }
I20260811 22:25:45.488894 31375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:45.489302 31374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8393da843d7148aaabe74dc2810a1bf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8393da843d7148aaabe74dc2810a1bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } } }
I20260811 22:25:45.489405 31374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:45.489841 31386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:45.493530 31386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:45.499806 31386 catalog_manager.cc:1347] Generated new cluster ID: 3698b6f9aba44e77b787abf04aaef03c
I20260811 22:25:45.499888 31386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:45.543141 31386 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:45.544539 31386 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:45.550231 31386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8393da843d7148aaabe74dc2810a1bf6: Generated new TSK 0
I20260811 22:25:45.551222 31386 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:45.569828 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31237
W20260811 22:25:45.585309 31213 connection.cc:535] client connection to 127.0.0.1:43291 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43291): Connection reset by peer (error 104)
2026-08-11T22:25:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1653 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:45 dist-test-slave-5bn2 krb5kdc[31409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:45 dist-test-slave-5bn2 krb5kdc[31409](info): set up 2 sockets
Aug 11 22:25:45 dist-test-slave-5bn2 krb5kdc[31409](info): commencing operation
krb5kdc: starting...
W20260811 22:25:46.759577 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:46 dist-test-slave-5bn2 krb5kdc[31409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:46.899458 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:47.016561 31427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:47.016872 31427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:47.016920 31427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:47.020980 31427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:47.021057 31427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:47.021076 31427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:47.021092 31427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:47.021106 31427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:47.026721 31427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:47.027880 31427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:47.028923 31427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:47.038120 31439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:47.038174 31437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:47.038174 31436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:47.038584 31427 server_base.cc:1034] running on GCE node
I20260811 22:25:47.039355 31427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:47.040642 31427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:47.041831 31427 hybrid_clock.cc:648] HybridClock initialized: now 1786487147041779 us; error 67 us; skew 500 ppm
Aug 11 22:25:47 dist-test-slave-5bn2 krb5kdc[31409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:47.044822 31427 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:47.045857 31427 webserver.cc:460] Webserver started at http://127.0.0.1:34001/ using document root <none> and password file <none>
I20260811 22:25:47.046484 31427 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:47.046564 31427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:47.046787 31427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:47.049007 31427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "f125b721dcf443bc8935a3f16a0c8da6"
format_stamp: "Formatted at 2026-08-11 22:25:47 on dist-test-slave-5bn2"
server_key: "43ce4228b31ef2853c708cde28e852f7"
server_key_iv: "baf97e8f81f731564b71bb3bb6e689c7"
server_key_version: "encryptionkey@0"
I20260811 22:25:47.049628 31427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "f125b721dcf443bc8935a3f16a0c8da6"
format_stamp: "Formatted at 2026-08-11 22:25:47 on dist-test-slave-5bn2"
server_key: "43ce4228b31ef2853c708cde28e852f7"
server_key_iv: "baf97e8f81f731564b71bb3bb6e689c7"
server_key_version: "encryptionkey@0"
I20260811 22:25:47.053819 31427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:47.056504 31455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:47.057994 31427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 22:25:47.058130 31427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "f125b721dcf443bc8935a3f16a0c8da6"
format_stamp: "Formatted at 2026-08-11 22:25:47 on dist-test-slave-5bn2"
server_key: "43ce4228b31ef2853c708cde28e852f7"
server_key_iv: "baf97e8f81f731564b71bb3bb6e689c7"
server_key_version: "encryptionkey@0"
I20260811 22:25:47.058243 31427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:47.063289 31427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:47.066761 31427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:47.066988 31427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:47.074239 31427 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33701
I20260811 22:25:47.074250 31545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33701 every 8 connection(s)
I20260811 22:25:47.075340 31427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:47.077621 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31427
I20260811 22:25:47.077772 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:47.078063 27124 external_mini_cluster.cc:1428] Setting key 69e468029934d8af165aa6f402c278dd
I20260811 22:25:47.078667 31547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:47.083992 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:47.079630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:41151 (local address 127.0.0.1:33701)
0811 22:25:47.080093 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:47.080111 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:47.080153 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:25:47.080848 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:47.080867 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:47.081203 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:47.081537 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:47.082472 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:47.082477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:47.082492 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:47.082504 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:47.082567 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:47.082588 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:47.082710 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:47.082817 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 22:25:47.083062 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":130,"threads_started":1}
I20260811 22:25:47.086072 31547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6: Bootstrap starting.
I20260811 22:25:47.087466 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:47.088760 31547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:47.089627 31547 log.cc:824] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:47.091454 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:47.089612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:41763 (local address 127.0.0.1:33701)
0811 22:25:47.089798 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:47.089803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:47.089972 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:25:47.090117 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:47.090120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:47.090156 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:47.090217 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:47.091076 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:47.091079 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:47.091082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:47.091095 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:47.091125 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:47.091138 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:47.091228 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:47.091286 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:25:47.091326 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:25:47.092275 31547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6: No bootstrap required, opened a new log
I20260811 22:25:47.096186 31547 raft_consensus.cc:348] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f125b721dcf443bc8935a3f16a0c8da6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } }
I20260811 22:25:47.096465 31547 raft_consensus.cc:374] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:47.096529 31547 raft_consensus.cc:729] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f125b721dcf443bc8935a3f16a0c8da6, State: Initialized, Role: FOLLOWER
I20260811 22:25:47.097011 31547 consensus_queue.cc:260] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f125b721dcf443bc8935a3f16a0c8da6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } }
I20260811 22:25:47.097146 31547 raft_consensus.cc:388] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:47.097214 31547 raft_consensus.cc:482] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:47.097311 31547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:47.098413 31547 raft_consensus.cc:504] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f125b721dcf443bc8935a3f16a0c8da6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } }
I20260811 22:25:47.098806 31547 leader_election.cc:302] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f125b721dcf443bc8935a3f16a0c8da6; no voters:
I20260811 22:25:47.099177 31547 leader_election.cc:288] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:47.099327 31567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:47.099581 31567 raft_consensus.cc:686] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [term 1 LEADER]: Becoming Leader. State: Replica: f125b721dcf443bc8935a3f16a0c8da6, State: Running, Role: LEADER
I20260811 22:25:47.099944 31567 consensus_queue.cc:237] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f125b721dcf443bc8935a3f16a0c8da6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } }
I20260811 22:25:47.100680 31547 sys_catalog.cc:564] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:47.101781 31569 sys_catalog.cc:455] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f125b721dcf443bc8935a3f16a0c8da6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f125b721dcf443bc8935a3f16a0c8da6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } } }
I20260811 22:25:47.101797 31572 sys_catalog.cc:455] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f125b721dcf443bc8935a3f16a0c8da6. Latest consensus state: current_term: 1 leader_uuid: "f125b721dcf443bc8935a3f16a0c8da6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f125b721dcf443bc8935a3f16a0c8da6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33701 } } }
I20260811 22:25:47.101939 31569 sys_catalog.cc:458] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:47.101939 31572 sys_catalog.cc:458] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:47.102356 31580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:47.105772 31580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:47.112309 31580 catalog_manager.cc:1347] Generated new cluster ID: 8a44066151634b09823e5eeacc149b37
I20260811 22:25:47.112414 31580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:47.127128 31580 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:47.127538 31580 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:47.150134 31580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f125b721dcf443bc8935a3f16a0c8da6: Generated new TSK 0
I20260811 22:25:47.151214 31580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:47.158435 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31427
2026-08-11T22:25:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:47 dist-test-slave-5bn2 krb5kdc[31602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:47 dist-test-slave-5bn2 krb5kdc[31602](info): set up 2 sockets
Aug 11 22:25:47 dist-test-slave-5bn2 krb5kdc[31602](info): commencing operation
krb5kdc: starting...
W20260811 22:25:48.364672 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:48 dist-test-slave-5bn2 krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:48.499114 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:48.617628 31624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:48.617949 31624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:48.618013 31624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:48.622201 31624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:48.622452 31624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:48.622519 31624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:48.622553 31624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:48.622576 31624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:48.628469 31624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:48.629688 31624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:48.630806 31624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:48.641069 31629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:48.641069 31630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:48.641069 31635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:48.641399 31624 server_base.cc:1034] running on GCE node
I20260811 22:25:48.642077 31624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:48.643378 31624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:48.644582 31624 hybrid_clock.cc:648] HybridClock initialized: now 1786487148644560 us; error 33 us; skew 500 ppm
Aug 11 22:25:48 dist-test-slave-5bn2 krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:48.647560 31624 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:48.648613 31624 webserver.cc:460] Webserver started at http://127.0.0.1:34143/ using document root <none> and password file <none>
I20260811 22:25:48.649256 31624 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:48.649322 31624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:48.649530 31624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:48.651859 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "f4f6e7e686b34a29bc8e6d63d0038f84"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-5bn2"
server_key: "584b1cc62a84fb68a26d5e43adfd1e51"
server_key_iv: "ecacbf84e22775a657176c71c26510fa"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.652513 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "f4f6e7e686b34a29bc8e6d63d0038f84"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-5bn2"
server_key: "584b1cc62a84fb68a26d5e43adfd1e51"
server_key_iv: "ecacbf84e22775a657176c71c26510fa"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.656627 31624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:25:48.659403 31646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:48.660800 31624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:48.660935 31624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "f4f6e7e686b34a29bc8e6d63d0038f84"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-5bn2"
server_key: "584b1cc62a84fb68a26d5e43adfd1e51"
server_key_iv: "ecacbf84e22775a657176c71c26510fa"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.661044 31624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:48.679869 31624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:48.683451 31624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:48.683638 31624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:48.691401 31624 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34293
I20260811 22:25:48.691414 31733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34293 every 8 connection(s)
I20260811 22:25:48.692559 31624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:48.696622 31734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:48.697611 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31624
I20260811 22:25:48.697734 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:48.698002 27124 external_mini_cluster.cc:1428] Setting key 726136ec00aed1428847746987d7347b
I20260811 22:25:48.703233 31734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84: Bootstrap starting.
I20260811 22:25:48.705699 31734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:48.706496 31734 log.cc:824] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:25:48 dist-test-slave-5bn2 krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:25:48.708775 31734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84: No bootstrap required, opened a new log
I20260811 22:25:48.712635 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.699554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:51539 (local address 127.0.0.1:34293)
0811 22:25:48.700072 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.700088 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.700127 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:25:48.700856 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.700880 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.701260 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.701606 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:48.702452 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.703558 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.704446 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.704935 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.705090 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.707927 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:48.707933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:48.707949 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:48.707979 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:48.709611 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.710157 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.710162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.710167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.710293 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.710628 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.710631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.710632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.711100 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:48.711306 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.711644 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.711864 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":161,"threads_started":1}
I20260811 22:25:48.712481 31734 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f6e7e686b34a29bc8e6d63d0038f84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260811 22:25:48.712733 31734 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:48.712780 31734 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f6e7e686b34a29bc8e6d63d0038f84, State: Initialized, Role: FOLLOWER
I20260811 22:25:48.713292 31734 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f6e7e686b34a29bc8e6d63d0038f84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260811 22:25:48.713459 31734 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:48.713526 31734 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:48.713620 31734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:48.714572 31734 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f6e7e686b34a29bc8e6d63d0038f84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260811 22:25:48.714987 31734 leader_election.cc:302] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4f6e7e686b34a29bc8e6d63d0038f84; no voters:
I20260811 22:25:48.715317 31734 leader_election.cc:288] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:48.715468 31742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:48.715727 31742 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [term 1 LEADER]: Becoming Leader. State: Replica: f4f6e7e686b34a29bc8e6d63d0038f84, State: Running, Role: LEADER
I20260811 22:25:48.715947 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:48.716068 31742 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f6e7e686b34a29bc8e6d63d0038f84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260811 22:25:48.716677 31734 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:48.717789 31745 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4f6e7e686b34a29bc8e6d63d0038f84. Latest consensus state: current_term: 1 leader_uuid: "f4f6e7e686b34a29bc8e6d63d0038f84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f6e7e686b34a29bc8e6d63d0038f84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } } }
I20260811 22:25:48.717924 31745 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:48.718246 31743 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4f6e7e686b34a29bc8e6d63d0038f84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f6e7e686b34a29bc8e6d63d0038f84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } } }
I20260811 22:25:48.718330 31743 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84 [sys.catalog]: This master's current role is: LEADER
W20260811 22:25:48.721662 31769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:25:48.721736 31769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:25:48.721866 31759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:48.722777 31759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:48.729780 31759 catalog_manager.cc:1347] Generated new cluster ID: 2478d3692f844068bed479d1a70b69e6
I20260811 22:25:48.729950 31759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:48.730147 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.718315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:39301 (local address 127.0.0.1:34293)
0811 22:25:48.718428 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.718432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.722055 (+ 3623us) server_negotiation.cc:407] Connection header received
0811 22:25:48.722195 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.722198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.722247 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.722311 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:48.723037 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.723562 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.724345 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.724579 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.724666 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.726296 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:48.726300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:48.726304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:48.726334 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:48.728043 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.728661 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.728665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.728667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.728766 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.729128 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.729132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.729133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.729364 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:48.729506 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.729863 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.729969 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:25:48.749540 31759 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:48.751266 31759 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:48.759493 31759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4f6e7e686b34a29bc8e6d63d0038f84: Generated new TSK 0
I20260811 22:25:48.760461 31759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:48.771420 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31624
W20260811 22:25:48.793401 31597 connection.cc:535] client connection to 127.0.0.1:34293 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34293): Connection reset by peer (error 104)
2026-08-11T22:25:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1615 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:48 dist-test-slave-5bn2 krb5kdc[31780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:48 dist-test-slave-5bn2 krb5kdc[31780](info): set up 2 sockets
Aug 11 22:25:48 dist-test-slave-5bn2 krb5kdc[31780](info): commencing operation
krb5kdc: starting...
W20260811 22:25:49.989655 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:50 dist-test-slave-5bn2 krb5kdc[31780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:50.125367 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:50.242255 31805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:50.242568 31805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:50.242610 31805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:50.246662 31805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:50.246744 31805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:50.246763 31805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:50.246778 31805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:50.246791 31805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:50.252417 31805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:50.253652 31805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:50.254792 31805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:50.263705 31813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:50.263706 31821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:50.263718 31818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:50.263906 31805 server_base.cc:1034] running on GCE node
I20260811 22:25:50.264830 31805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:50.266098 31805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:50.267271 31805 hybrid_clock.cc:648] HybridClock initialized: now 1786487150267250 us; error 38 us; skew 500 ppm
Aug 11 22:25:50 dist-test-slave-5bn2 krb5kdc[31780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487150, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:50.270298 31805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:50.271333 31805 webserver.cc:460] Webserver started at http://127.0.0.1:37663/ using document root <none> and password file <none>
I20260811 22:25:50.271909 31805 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:50.271978 31805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:50.272163 31805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:50.274356 31805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "f608eaea717542ab9cd9f99a81a0402f"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-5bn2"
server_key: "fff1be3fbcec61befabe83e200ebb58f"
server_key_iv: "12195103cb42e90b4dc8f5faee537d1a"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.274992 31805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "f608eaea717542ab9cd9f99a81a0402f"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-5bn2"
server_key: "fff1be3fbcec61befabe83e200ebb58f"
server_key_iv: "12195103cb42e90b4dc8f5faee537d1a"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.279178 31805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:50.281934 31834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:50.283656 31805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:50.283808 31805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "f608eaea717542ab9cd9f99a81a0402f"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-5bn2"
server_key: "fff1be3fbcec61befabe83e200ebb58f"
server_key_iv: "12195103cb42e90b4dc8f5faee537d1a"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.283929 31805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:50.314949 31805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:50.318428 31805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:50.318632 31805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:50.325744 31805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39491
I20260811 22:25:50.325762 31926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39491 every 8 connection(s)
I20260811 22:25:50.326771 31805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:50.329833 31929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:50.334015 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 31805
I20260811 22:25:50.334165 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:50.334462 27124 external_mini_cluster.cc:1428] Setting key d5db941596c64b94d094a9c82ac19fa5
I20260811 22:25:50.336160 31929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f: Bootstrap starting.
I20260811 22:25:50.339119 31929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:50.340063 31929 log.cc:824] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:50.342411 31929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f: No bootstrap required, opened a new log
I20260811 22:25:50.345269 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:50.336031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:48891 (local address 127.0.0.1:39491)
0811 22:25:50.336488 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:50.336511 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:50.336558 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:25:50.337438 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:50.337466 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:50.337942 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:50.338338 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:50.338780 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.340321 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.341415 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.342099 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.342260 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.343080 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:50.343087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:50.343105 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:50.343651 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:50.343734 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:50.343766 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:50.343905 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:50.344115 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 22:25:50.344263 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":125,"threads_started":1}
I20260811 22:25:50.346729 31929 raft_consensus.cc:348] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f608eaea717542ab9cd9f99a81a0402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260811 22:25:50.347010 31929 raft_consensus.cc:374] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:50.347069 31929 raft_consensus.cc:729] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f608eaea717542ab9cd9f99a81a0402f, State: Initialized, Role: FOLLOWER
I20260811 22:25:50.347546 31929 consensus_queue.cc:260] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f608eaea717542ab9cd9f99a81a0402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260811 22:25:50.347681 31929 raft_consensus.cc:388] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:50.347749 31929 raft_consensus.cc:482] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:50.347848 31929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:50.348649 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:50.349021 31929 raft_consensus.cc:504] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f608eaea717542ab9cd9f99a81a0402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260811 22:25:50.349368 31929 leader_election.cc:302] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f608eaea717542ab9cd9f99a81a0402f; no voters:
I20260811 22:25:50.349712 31929 leader_election.cc:288] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:50.349892 31935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:50.350207 31935 raft_consensus.cc:686] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [term 1 LEADER]: Becoming Leader. State: Replica: f608eaea717542ab9cd9f99a81a0402f, State: Running, Role: LEADER
I20260811 22:25:50.350641 31935 consensus_queue.cc:237] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f608eaea717542ab9cd9f99a81a0402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260811 22:25:50.351188 31929 sys_catalog.cc:564] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:50.352337 31935 sys_catalog.cc:455] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f608eaea717542ab9cd9f99a81a0402f. Latest consensus state: current_term: 1 leader_uuid: "f608eaea717542ab9cd9f99a81a0402f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f608eaea717542ab9cd9f99a81a0402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } } }
I20260811 22:25:50.352569 31935 sys_catalog.cc:458] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:50.352816 31942 sys_catalog.cc:455] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f608eaea717542ab9cd9f99a81a0402f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f608eaea717542ab9cd9f99a81a0402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } } }
I20260811 22:25:50.353008 31942 sys_catalog.cc:458] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:50.353231 31961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:50.359217 31961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:50.359241 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:50.352118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:47815 (local address 127.0.0.1:39491)
0811 22:25:50.352235 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:50.352239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:50.353388 (+ 1149us) server_negotiation.cc:407] Connection header received
0811 22:25:50.353640 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:50.353643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:50.353699 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:50.353750 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:50.354997 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.355885 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.357117 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.357454 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.357565 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.358506 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:50.358510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:50.358513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:50.358528 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:50.358573 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:50.358588 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:50.358674 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:50.359055 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 22:25:50.359095 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:25:50.366575 31961 catalog_manager.cc:1347] Generated new cluster ID: f0c85dfe3fec4c2b8b29f58b0956b89d
I20260811 22:25:50.366734 31961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:50.390771 31961 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:50.391911 31961 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:50.401398 31961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f608eaea717542ab9cd9f99a81a0402f: Generated new TSK 0
I20260811 22:25:50.402281 31961 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:50.418470 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 31805
W20260811 22:25:50.430050 31776 connection.cc:535] client connection to 127.0.0.1:39491 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39491): Connection reset by peer (error 104)
2026-08-11T22:25:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1636 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:50 dist-test-slave-5bn2 krb5kdc[31982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:50 dist-test-slave-5bn2 krb5kdc[31982](info): set up 2 sockets
Aug 11 22:25:50 dist-test-slave-5bn2 krb5kdc[31982](info): commencing operation
krb5kdc: starting...
W20260811 22:25:51.622831 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:51 dist-test-slave-5bn2 krb5kdc[31982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:25:51.758123 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:51.874294 32016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:51.874629 32016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:51.874691 32016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:51.878899 32016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:51.878983 32016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:51.879009 32016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:51.879040 32016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:51.879068 32016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:51.885244 32016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32797
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:51.886756 32016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:51.888240 32016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:51.897565 32023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:51.897581 32022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:51.897758 32016 server_base.cc:1034] running on GCE node
W20260811 22:25:51.897993 32025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:51.898705 32016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:51.900038 32016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:51.901221 32016 hybrid_clock.cc:648] HybridClock initialized: now 1786487151901198 us; error 43 us; skew 500 ppm
Aug 11 22:25:51 dist-test-slave-5bn2 krb5kdc[31982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:51.904250 32016 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:25:51.905370 32016 webserver.cc:460] Webserver started at http://127.0.0.1:39623/ using document root <none> and password file <none>
I20260811 22:25:51.905961 32016 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:51.906030 32016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:51.906212 32016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:51.908473 32016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "17fb60f62058485bb7108d6563dd7a6b"
format_stamp: "Formatted at 2026-08-11 22:25:51 on dist-test-slave-5bn2"
server_key: "e457264144990934e82c1380b2dc1915"
server_key_iv: "6f87987274306260206a0f7059b03f48"
server_key_version: "encryptionkey@0"
I20260811 22:25:51.909142 32016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "17fb60f62058485bb7108d6563dd7a6b"
format_stamp: "Formatted at 2026-08-11 22:25:51 on dist-test-slave-5bn2"
server_key: "e457264144990934e82c1380b2dc1915"
server_key_iv: "6f87987274306260206a0f7059b03f48"
server_key_version: "encryptionkey@0"
I20260811 22:25:51.913352 32016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:51.916165 32041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:51.917704 32016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:51.917855 32016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "17fb60f62058485bb7108d6563dd7a6b"
format_stamp: "Formatted at 2026-08-11 22:25:51 on dist-test-slave-5bn2"
server_key: "e457264144990934e82c1380b2dc1915"
server_key_iv: "6f87987274306260206a0f7059b03f48"
server_key_version: "encryptionkey@0"
I20260811 22:25:51.917970 32016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:51.922979 32016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:51.926374 32016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:51.926538 32016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:51.933751 32016 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32797
I20260811 22:25:51.933765 32135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32797 every 8 connection(s)
I20260811 22:25:51.934747 32016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:51.935845 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 32016
I20260811 22:25:51.935954 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:51.936232 27124 external_mini_cluster.cc:1428] Setting key ce7d0c6b6eb3231ec20639aa98f6333f
I20260811 22:25:51.937968 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:51.941947 32139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:51.937746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:51799 (local address 127.0.0.1:32797)
0811 22:25:51.938171 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:51.938190 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:51.938233 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:25:51.939083 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:51.939107 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:51.939473 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:51.939845 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:51.940509 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:51.940515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:51.940529 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:51.940540 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:51.940602 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:51.940630 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:51.940767 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:51.940838 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 22:25:51.941009 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":110,"threads_started":1}
I20260811 22:25:51.944747 32136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b: Bootstrap starting.
I20260811 22:25:51.944782 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:51.947615 32136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:51.948525 32136 log.cc:824] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:51.948789 32139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:51.947102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:35373 (local address 127.0.0.1:32797)
0811 22:25:51.947226 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:51.947230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:51.947445 (+ 215us) server_negotiation.cc:407] Connection header received
0811 22:25:51.947581 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:51.947583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:51.947616 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:51.947682 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:51.948397 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:51.948400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:51.948402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:51.948412 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:51.948439 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:51.948448 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:51.948535 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:51.948629 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 22:25:51.948669 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:25:51.950592 32136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b: No bootstrap required, opened a new log
I20260811 22:25:51.954246 32136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fb60f62058485bb7108d6563dd7a6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32797 } }
I20260811 22:25:51.954517 32136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:51.954572 32136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fb60f62058485bb7108d6563dd7a6b, State: Initialized, Role: FOLLOWER
I20260811 22:25:51.955081 32136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fb60f62058485bb7108d6563dd7a6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32797 } }
I20260811 22:25:51.955263 32136 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:51.955372 32136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:51.955489 32136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:51.956588 32136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fb60f62058485bb7108d6563dd7a6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32797 } }
I20260811 22:25:51.956923 32136 leader_election.cc:302] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17fb60f62058485bb7108d6563dd7a6b; no voters:
I20260811 22:25:51.957260 32136 leader_election.cc:288] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:51.957373 32148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:51.957599 32148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [term 1 LEADER]: Becoming Leader. State: Replica: 17fb60f62058485bb7108d6563dd7a6b, State: Running, Role: LEADER
I20260811 22:25:51.957973 32148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fb60f62058485bb7108d6563dd7a6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32797 } }
I20260811 22:25:51.958575 32136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:51.959678 32150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fb60f62058485bb7108d6563dd7a6b. Latest consensus state: current_term: 1 leader_uuid: "17fb60f62058485bb7108d6563dd7a6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fb60f62058485bb7108d6563dd7a6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32797 } } }
I20260811 22:25:51.959815 32150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:51.960201 32160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:51.961226 32149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17fb60f62058485bb7108d6563dd7a6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fb60f62058485bb7108d6563dd7a6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32797 } } }
I20260811 22:25:51.961330 32149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:51.963234 32160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:51.968766 32160 catalog_manager.cc:1347] Generated new cluster ID: 365bec2e8e7a40a6bbb2459dd1145d57
I20260811 22:25:51.968847 32160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:51.978382 32160 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:51.978765 32160 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:51.985903 32160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17fb60f62058485bb7108d6563dd7a6b: Generated new TSK 0
I20260811 22:25:51.986892 32160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:51.996707 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 32016
2026-08-11T22:25:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1589 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:52 dist-test-slave-5bn2 krb5kdc[32182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:52 dist-test-slave-5bn2 krb5kdc[32182](info): set up 2 sockets
Aug 11 22:25:52 dist-test-slave-5bn2 krb5kdc[32182](info): commencing operation
krb5kdc: starting...
W20260811 22:25:53.186463 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:53 dist-test-slave-5bn2 krb5kdc[32182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:25:53.322012 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40695
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:53.439455 32205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:53.439759 32205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:53.439821 32205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:53.443873 32205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:53.443951 32205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:53.443977 32205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:53.443998 32205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:53.444023 32205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:53.449496 32205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:53.450667 32205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:53.451800 32205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:53.460479 32213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:53.460537 32214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:53.460547 32217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:53.460665 32205 server_base.cc:1034] running on GCE node
I20260811 22:25:53.461395 32205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:53.462558 32205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:53.463739 32205 hybrid_clock.cc:648] HybridClock initialized: now 1786487153463711 us; error 43 us; skew 500 ppm
Aug 11 22:25:53 dist-test-slave-5bn2 krb5kdc[32182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487153, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:53.466665 32205 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:53.467654 32205 webserver.cc:460] Webserver started at http://127.26.125.62:32773/ using document root <none> and password file <none>
I20260811 22:25:53.468235 32205 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:53.468305 32205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:53.468523 32205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:53.470721 32205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "9dec8608408749ec9e2a122705cd78ef"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-5bn2"
server_key: "c9b1d60a4d29d88097090c27f43071d6"
server_key_iv: "725e09e17addaa1b5b79a6fa2f718b6a"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.471349 32205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "9dec8608408749ec9e2a122705cd78ef"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-5bn2"
server_key: "c9b1d60a4d29d88097090c27f43071d6"
server_key_iv: "725e09e17addaa1b5b79a6fa2f718b6a"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.475183 32205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:53.477669 32226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:53.479022 32205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:25:53.479152 32205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "9dec8608408749ec9e2a122705cd78ef"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-5bn2"
server_key: "c9b1d60a4d29d88097090c27f43071d6"
server_key_iv: "725e09e17addaa1b5b79a6fa2f718b6a"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.479252 32205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:53.491498 32205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:53.494729 32205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:53.494925 32205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:53.502100 32205 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:40695
I20260811 22:25:53.502147 32322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:40695 every 8 connection(s)
I20260811 22:25:53.503178 32205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:53.506453 32323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:53.509833 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 32205
I20260811 22:25:53.510005 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:53.510303 27124 external_mini_cluster.cc:1428] Setting key e39bfc206703f2aabd23260dde1a5bfc
I20260811 22:25:53.513072 32323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef: Bootstrap starting.
I20260811 22:25:53.515867 32323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:53.516891 32323 log.cc:824] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:53.519793 32323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef: No bootstrap required, opened a new log
Aug 11 22:25:53 dist-test-slave-5bn2 krb5kdc[32182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:53.523258 32323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dec8608408749ec9e2a122705cd78ef" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40695 } }
I20260811 22:25:53.523569 32323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:53.523639 32323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dec8608408749ec9e2a122705cd78ef, State: Initialized, Role: FOLLOWER
I20260811 22:25:53.524195 32323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dec8608408749ec9e2a122705cd78ef" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40695 } }
I20260811 22:25:53.524363 32323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:53.524442 32323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:53.524557 32323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:53.525604 32323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dec8608408749ec9e2a122705cd78ef" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40695 } }
I20260811 22:25:53.526077 32323 leader_election.cc:302] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dec8608408749ec9e2a122705cd78ef; no voters:
I20260811 22:25:53.526499 32323 leader_election.cc:288] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:53.526624 32333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:53.526818 32333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [term 1 LEADER]: Becoming Leader. State: Replica: 9dec8608408749ec9e2a122705cd78ef, State: Running, Role: LEADER
I20260811 22:25:53.527302 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:53.511942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40029 (local address 127.26.125.62:40695)
0811 22:25:53.512404 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:53.512429 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:53.512468 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:25:53.513117 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:53.513135 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:53.513539 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:53.513835 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:53.514794 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.515910 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.517160 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.517853 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.517991 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:53.521319 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:53.521326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:53.521345 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:53.521395 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:53.523820 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:53.524433 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:53.524440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:53.524447 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:53.524612 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:53.524945 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:53.524948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:53.524950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:53.525380 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:53.525606 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:53.525944 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 22:25:53.526186 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":129,"threads_started":1}
I20260811 22:25:53.527300 32333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dec8608408749ec9e2a122705cd78ef" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40695 } }
I20260811 22:25:53.528083 32323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:53.530649 32336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dec8608408749ec9e2a122705cd78ef. Latest consensus state: current_term: 1 leader_uuid: "9dec8608408749ec9e2a122705cd78ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dec8608408749ec9e2a122705cd78ef" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40695 } } }
I20260811 22:25:53.530900 32336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:53.531106 32335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dec8608408749ec9e2a122705cd78ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dec8608408749ec9e2a122705cd78ef" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40695 } } }
I20260811 22:25:53.531713 32335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:53.531723 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 22:25:53.534328 32365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:25:53.534435 32365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:25:53.534507 32364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:53.535506 32364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:53.543089 32364 catalog_manager.cc:1347] Generated new cluster ID: 5b584ca7192342e0a461fa7eda3fd2f2
I20260811 22:25:53.543162 32364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:53.547009 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:53.535326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34789 (local address 127.26.125.62:40695)
0811 22:25:53.535437 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:53.535441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:53.537904 (+ 2463us) server_negotiation.cc:407] Connection header received
0811 22:25:53.538065 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:53.538069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:53.538122 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:53.538208 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:53.539170 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.540196 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.541389 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.541638 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.541731 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:53.543394 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:53.543398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:53.543401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:53.543427 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:53.545090 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:53.545677 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:53.545680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:53.545682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:53.545762 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:53.546106 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:53.546109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:53.546111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:53.546299 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:53.546446 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:53.546730 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 22:25:53.546824 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:25:53.580629 32364 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:53.582078 32364 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:53.594141 32364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dec8608408749ec9e2a122705cd78ef: Generated new TSK 0
I20260811 22:25:53.595171 32364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:53.600320 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 32205
W20260811 22:25:53.613312 32178 connection.cc:535] client connection to 127.26.125.62:40695 recv error: Network error: failed to read from TLS socket (remote: 127.26.125.62:40695): Connection reset by peer (error 104)
2026-08-11T22:25:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1593 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:53 dist-test-slave-5bn2 krb5kdc[32383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:53 dist-test-slave-5bn2 krb5kdc[32383](info): set up 2 sockets
Aug 11 22:25:53 dist-test-slave-5bn2 krb5kdc[32383](info): commencing operation
krb5kdc: starting...
W20260811 22:25:54.801966 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:54 dist-test-slave-5bn2 krb5kdc[32383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:25:54.942386 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:37455
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:37455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:55.060904 32409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:55.061208 32409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:55.061250 32409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:55.065311 32409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:55.065389 32409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:55.065408 32409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:55.065424 32409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:55.065438 32409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:55.071050 32409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:37455
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:37455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:55.072223 32409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:55.073328 32409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:55.082772 32423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:55.082775 32421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:55.082775 32417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:55.082834 32409 server_base.cc:1034] running on GCE node
I20260811 22:25:55.083707 32409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:55.084923 32409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:55.086122 32409 hybrid_clock.cc:648] HybridClock initialized: now 1786487155086096 us; error 37 us; skew 500 ppm
Aug 11 22:25:55 dist-test-slave-5bn2 krb5kdc[32383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487155, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:55.089164 32409 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:55.090240 32409 webserver.cc:460] Webserver started at http://127.26.125.62:39171/ using document root <none> and password file <none>
I20260811 22:25:55.090837 32409 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:55.090956 32409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:55.091210 32409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:55.093570 32409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "65f77c5bbef148bb89d3c43d9a78a814"
format_stamp: "Formatted at 2026-08-11 22:25:55 on dist-test-slave-5bn2"
server_key: "04ffcbaffb6ec3ca247bd881228a1d04"
server_key_iv: "4321d1eb13bb73eb7f015b5aacfaaa30"
server_key_version: "encryptionkey@0"
I20260811 22:25:55.094194 32409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "65f77c5bbef148bb89d3c43d9a78a814"
format_stamp: "Formatted at 2026-08-11 22:25:55 on dist-test-slave-5bn2"
server_key: "04ffcbaffb6ec3ca247bd881228a1d04"
server_key_iv: "4321d1eb13bb73eb7f015b5aacfaaa30"
server_key_version: "encryptionkey@0"
I20260811 22:25:55.098640 32409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 22:25:55.101590 32431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:55.103140 32409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:55.103295 32409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "65f77c5bbef148bb89d3c43d9a78a814"
format_stamp: "Formatted at 2026-08-11 22:25:55 on dist-test-slave-5bn2"
server_key: "04ffcbaffb6ec3ca247bd881228a1d04"
server_key_iv: "4321d1eb13bb73eb7f015b5aacfaaa30"
server_key_version: "encryptionkey@0"
I20260811 22:25:55.103420 32409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:55.125852 32409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:55.129628 32409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:55.129860 32409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:55.137854 32409 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:37455
I20260811 22:25:55.137864 32542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:37455 every 8 connection(s)
I20260811 22:25:55.139019 32409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:55.140846 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 32409
I20260811 22:25:55.141005 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:55.141280 27124 external_mini_cluster.cc:1428] Setting key 2ed5e185d144e9e00e51f2ab08a0372e
I20260811 22:25:55.142625 32545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:55.150658 32545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814: Bootstrap starting.
I20260811 22:25:55.152403 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:55.142761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38549 (local address 127.26.125.62:37455)
0811 22:25:55.143276 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:55.143298 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:55.143344 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:25:55.144222 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:55.144247 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:55.144764 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:55.145183 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:55.145652 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:55.147254 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:55.148314 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:55.149088 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:55.149284 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:55.150209 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:55.150215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:55.150234 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:55.150779 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:55.150887 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:55.150924 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:55.151060 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:55.151250 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 22:25:55.151411 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":137,"threads_started":1}
I20260811 22:25:55.153501 32545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:55.154431 32545 log.cc:824] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:55.155658 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:55.156625 32545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814: No bootstrap required, opened a new log
I20260811 22:25:55.160012 32545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f77c5bbef148bb89d3c43d9a78a814" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37455 } }
I20260811 22:25:55.160286 32545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:55.160348 32545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f77c5bbef148bb89d3c43d9a78a814, State: Initialized, Role: FOLLOWER
I20260811 22:25:55.160854 32545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f77c5bbef148bb89d3c43d9a78a814" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37455 } }
I20260811 22:25:55.161042 32545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:55.161121 32545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:55.161239 32545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:55.162238 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:55.157673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39765 (local address 127.26.125.62:37455)
0811 22:25:55.157780 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:55.157784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:55.157997 (+ 213us) server_negotiation.cc:407] Connection header received
0811 22:25:55.158112 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:55.158115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:55.158156 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:55.158218 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:55.158510 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:55.159429 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:55.160262 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:55.160553 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:55.160651 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:55.161773 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:55.161776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:55.161778 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:55.161792 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:55.161821 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:55.161833 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:55.161922 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:55.162057 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 22:25:55.162093 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:25:55.162595 32545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f77c5bbef148bb89d3c43d9a78a814" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37455 } }
I20260811 22:25:55.163031 32545 leader_election.cc:302] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65f77c5bbef148bb89d3c43d9a78a814; no voters:
I20260811 22:25:55.163455 32545 leader_election.cc:288] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:55.163614 32567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:55.163836 32567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [term 1 LEADER]: Becoming Leader. State: Replica: 65f77c5bbef148bb89d3c43d9a78a814, State: Running, Role: LEADER
I20260811 22:25:55.164211 32567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f77c5bbef148bb89d3c43d9a78a814" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37455 } }
I20260811 22:25:55.164793 32545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:55.165937 32568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65f77c5bbef148bb89d3c43d9a78a814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f77c5bbef148bb89d3c43d9a78a814" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37455 } } }
I20260811 22:25:55.166078 32568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:55.166352 32569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f77c5bbef148bb89d3c43d9a78a814. Latest consensus state: current_term: 1 leader_uuid: "65f77c5bbef148bb89d3c43d9a78a814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f77c5bbef148bb89d3c43d9a78a814" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37455 } } }
I20260811 22:25:55.166441 32569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:55.166492 32580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:55.169904 32580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:55.176651 32580 catalog_manager.cc:1347] Generated new cluster ID: baec7492c1a04bcf8b5d88c10ba1d220
I20260811 22:25:55.176749 32580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:55.190335 32580 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:55.191860 32580 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:55.199889 32580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65f77c5bbef148bb89d3c43d9a78a814: Generated new TSK 0
I20260811 22:25:55.200872 32580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:55.208947 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 32409
W20260811 22:25:55.228081 32380 connection.cc:535] client connection to 127.26.125.62:37455 recv error: Network error: failed to read from TLS socket (remote: 127.26.125.62:37455): Connection reset by peer (error 104)
2026-08-11T22:25:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:55 dist-test-slave-5bn2 krb5kdc[32602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:55 dist-test-slave-5bn2 krb5kdc[32602](info): set up 2 sockets
Aug 11 22:25:55 dist-test-slave-5bn2 krb5kdc[32602](info): commencing operation
krb5kdc: starting...
W20260811 22:25:56.401741 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:56 dist-test-slave-5bn2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:25:56.538195 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:43499
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:38323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:43499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:56.655690 32622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:56.655992 32622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:56.656040 32622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:56.660110 32622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:56.660192 32622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:56.660209 32622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:56.660225 32622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:56.660238 32622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:56.665971 32622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:38323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:43499
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:43499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:56.667522 32622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:56.668927 32622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:56.677914 32633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:56.678118 32622 server_base.cc:1034] running on GCE node
W20260811 22:25:56.677914 32634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:56.677919 32640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:56.678840 32622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:56.680142 32622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:56.681320 32622 hybrid_clock.cc:648] HybridClock initialized: now 1786487156681269 us; error 65 us; skew 500 ppm
Aug 11 22:25:56 dist-test-slave-5bn2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:56.684237 32622 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:56.685257 32622 webserver.cc:460] Webserver started at http://127.26.125.62:36991/ using document root <none> and password file <none>
I20260811 22:25:56.685834 32622 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:56.685905 32622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:56.686132 32622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:56.688301 32622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "4f002147c1ce4ff789942253795ea04b"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-5bn2"
server_key: "a0436a1d52f2016ed85ce6897a081023"
server_key_iv: "905ef827d99f90bf4e91b18aa9e58600"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.688891 32622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "4f002147c1ce4ff789942253795ea04b"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-5bn2"
server_key: "a0436a1d52f2016ed85ce6897a081023"
server_key_iv: "905ef827d99f90bf4e91b18aa9e58600"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.692813 32622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:56.695502 32650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.696873 32622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:56.697026 32622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "4f002147c1ce4ff789942253795ea04b"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-5bn2"
server_key: "a0436a1d52f2016ed85ce6897a081023"
server_key_iv: "905ef827d99f90bf4e91b18aa9e58600"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.697136 32622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:56.702044 32622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:56.705523 32622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:56.705724 32622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:56.712910 32622 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:43499
I20260811 22:25:56.712960 32748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:43499 every 8 connection(s)
I20260811 22:25:56.713900 32622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:56.715780 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 32622
I20260811 22:25:56.715924 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:56.716210 27124 external_mini_cluster.cc:1428] Setting key 8a69403778d82b44f276cca350223a09
I20260811 22:25:56.717190 32750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:56.721894 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:56.717684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46797 (local address 127.26.125.62:43499)
0811 22:25:56.718140 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:56.718160 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:56.718204 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:25:56.719075 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:56.719099 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:56.719471 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:56.719844 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:56.720492 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:56.720498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:56.720513 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:56.720524 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:56.720587 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:56.720612 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:56.720755 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:56.720825 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 22:25:56.720974 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":124,"threads_started":1}
I20260811 22:25:56.724069 32750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b: Bootstrap starting.
I20260811 22:25:56.724824 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:56.726907 32750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:56.727768 32750 log.cc:824] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:56.728785 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:56.727151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60327 (local address 127.26.125.62:43499)
0811 22:25:56.727265 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:56.727269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:56.727457 (+ 188us) server_negotiation.cc:407] Connection header received
0811 22:25:56.727587 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:56.727590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:56.727626 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:56.727710 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:56.728354 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:56.728366 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:56.728368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:56.728378 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:56.728402 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:56.728412 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:56.728478 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:56.728632 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 22:25:56.728670 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:25:56.730005 32750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b: No bootstrap required, opened a new log
I20260811 22:25:56.733477 32750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f002147c1ce4ff789942253795ea04b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43499 } }
I20260811 22:25:56.733711 32750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:56.733752 32750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f002147c1ce4ff789942253795ea04b, State: Initialized, Role: FOLLOWER
I20260811 22:25:56.734138 32750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f002147c1ce4ff789942253795ea04b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43499 } }
I20260811 22:25:56.734282 32750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:56.734352 32750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:56.734470 32750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:56.735455 32750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f002147c1ce4ff789942253795ea04b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43499 } }
I20260811 22:25:56.735785 32750 leader_election.cc:302] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f002147c1ce4ff789942253795ea04b; no voters:
I20260811 22:25:56.736090 32750 leader_election.cc:288] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:56.736223 300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:56.736433 300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [term 1 LEADER]: Becoming Leader. State: Replica: 4f002147c1ce4ff789942253795ea04b, State: Running, Role: LEADER
I20260811 22:25:56.736773 300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f002147c1ce4ff789942253795ea04b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43499 } }
I20260811 22:25:56.737305 32750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:56.738411 302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f002147c1ce4ff789942253795ea04b. Latest consensus state: current_term: 1 leader_uuid: "4f002147c1ce4ff789942253795ea04b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f002147c1ce4ff789942253795ea04b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43499 } } }
I20260811 22:25:56.738536 302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:56.738885 301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f002147c1ce4ff789942253795ea04b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f002147c1ce4ff789942253795ea04b" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 43499 } } }
I20260811 22:25:56.738951 313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:56.739675 301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:56.741864 313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:56.747409 313 catalog_manager.cc:1347] Generated new cluster ID: 6344a5af68204e1ab4bc5c343a097df4
I20260811 22:25:56.747488 313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:56.784723 313 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:56.785117 313 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:56.796672 313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f002147c1ce4ff789942253795ea04b: Generated new TSK 0
I20260811 22:25:56.797492 313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:56.808408 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 32622
2026-08-11T22:25:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1598 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:56 dist-test-slave-5bn2 krb5kdc[335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:56 dist-test-slave-5bn2 krb5kdc[335](info): set up 2 sockets
Aug 11 22:25:56 dist-test-slave-5bn2 krb5kdc[335](info): commencing operation
krb5kdc: starting...
W20260811 22:25:58.004009 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:58 dist-test-slave-5bn2 krb5kdc[335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:25:58.140689 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36877
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:43047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:58.257249 360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:58.257577 360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:58.257644 360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:58.261906 360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:58.262006 360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:58.262036 360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:58.262058 360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:58.262075 360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:58.267798 360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:43047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:58.269006 360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:58.270112 360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:58.279062 369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:58.279026 368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:58.279048 371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:58.279237 360 server_base.cc:1034] running on GCE node
I20260811 22:25:58.279991 360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:58.281141 360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:58.282315 360 hybrid_clock.cc:648] HybridClock initialized: now 1786487158282284 us; error 45 us; skew 500 ppm
Aug 11 22:25:58 dist-test-slave-5bn2 krb5kdc[335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487158, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:58.285394 360 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:58.286499 360 webserver.cc:460] Webserver started at http://127.26.125.62:38067/ using document root <none> and password file <none>
I20260811 22:25:58.287269 360 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:58.287353 360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:58.287604 360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:58.289852 360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "01ce37abcc5a4474b4f6635e3c997849"
format_stamp: "Formatted at 2026-08-11 22:25:58 on dist-test-slave-5bn2"
server_key: "8a6b206eec94084d99c350a4473fec18"
server_key_iv: "fd81b40af69f7b61e5faecf1ac99ced6"
server_key_version: "encryptionkey@0"
I20260811 22:25:58.290536 360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "01ce37abcc5a4474b4f6635e3c997849"
format_stamp: "Formatted at 2026-08-11 22:25:58 on dist-test-slave-5bn2"
server_key: "8a6b206eec94084d99c350a4473fec18"
server_key_iv: "fd81b40af69f7b61e5faecf1ac99ced6"
server_key_version: "encryptionkey@0"
I20260811 22:25:58.294760 360 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 22:25:58.297638 380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:58.299234 360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:58.299376 360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "01ce37abcc5a4474b4f6635e3c997849"
format_stamp: "Formatted at 2026-08-11 22:25:58 on dist-test-slave-5bn2"
server_key: "8a6b206eec94084d99c350a4473fec18"
server_key_iv: "fd81b40af69f7b61e5faecf1ac99ced6"
server_key_version: "encryptionkey@0"
I20260811 22:25:58.299502 360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:58.322259 360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:58.325806 360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:58.326009 360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:58.333323 360 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:36877
I20260811 22:25:58.333346 474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:36877 every 8 connection(s)
I20260811 22:25:58.334349 360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:58.337292 475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:58.339031 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 360
I20260811 22:25:58.339174 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:58.339432 27124 external_mini_cluster.cc:1428] Setting key a0410a44c6be2267b3e97a8e6d15c632
I20260811 22:25:58.343933 475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849: Bootstrap starting.
I20260811 22:25:58.346626 475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:58.347599 475 log.cc:824] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:25:58 dist-test-slave-5bn2 krb5kdc[335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:25:58.349891 475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849: No bootstrap required, opened a new log
I20260811 22:25:58.354489 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.340816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38855 (local address 127.26.125.62:36877)
0811 22:25:58.341241 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.341257 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.341296 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:25:58.341966 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.341984 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.342353 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.342655 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.343639 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.344715 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.345930 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.346434 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.346569 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.349355 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.349363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.349381 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.349414 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.351371 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.351785 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.351791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.351798 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.351955 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.352175 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.352178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.352180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.352742 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.352954 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.353141 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.353459 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
I20260811 22:25:58.354526 475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ce37abcc5a4474b4f6635e3c997849" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36877 } }
I20260811 22:25:58.354794 475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:58.354874 475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ce37abcc5a4474b4f6635e3c997849, State: Initialized, Role: FOLLOWER
I20260811 22:25:58.355372 475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ce37abcc5a4474b4f6635e3c997849" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36877 } }
I20260811 22:25:58.355531 475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:58.355595 475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:58.355706 475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:58.356856 475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ce37abcc5a4474b4f6635e3c997849" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36877 } }
I20260811 22:25:58.357208 475 leader_election.cc:302] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01ce37abcc5a4474b4f6635e3c997849; no voters:
I20260811 22:25:58.357583 475 leader_election.cc:288] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:58.357712 483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:58.357852 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:58.357944 483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [term 1 LEADER]: Becoming Leader. State: Replica: 01ce37abcc5a4474b4f6635e3c997849, State: Running, Role: LEADER
I20260811 22:25:58.358271 483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ce37abcc5a4474b4f6635e3c997849" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36877 } }
I20260811 22:25:58.359010 475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:58.359449 486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01ce37abcc5a4474b4f6635e3c997849. Latest consensus state: current_term: 1 leader_uuid: "01ce37abcc5a4474b4f6635e3c997849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ce37abcc5a4474b4f6635e3c997849" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36877 } } }
I20260811 22:25:58.359565 486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:58.360213 485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01ce37abcc5a4474b4f6635e3c997849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ce37abcc5a4474b4f6635e3c997849" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36877 } } }
I20260811 22:25:58.360323 485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849 [sys.catalog]: This master's current role is: LEADER
W20260811 22:25:58.364437 522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:25:58.364531 522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:25:58.364596 496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:58.365527 496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:58.369876 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.360776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53793 (local address 127.26.125.62:36877)
0811 22:25:58.360890 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.360895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.361797 (+ 902us) server_negotiation.cc:407] Connection header received
0811 22:25:58.361888 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.361891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.361938 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.362021 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.362645 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.363441 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.364600 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.364935 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.365030 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.366785 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.366788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.366791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.366817 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.368332 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.368727 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.368731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.368733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.368810 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.369058 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.369061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.369063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.369243 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.369355 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.369592 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.369705 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 22:25:58.372694 496 catalog_manager.cc:1347] Generated new cluster ID: b3acaa2ed85a488aab34339679a9b0c9
I20260811 22:25:58.372752 496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:58.380321 496 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:58.381611 496 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:58.388850 496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01ce37abcc5a4474b4f6635e3c997849: Generated new TSK 0
I20260811 22:25:58.389699 496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:58.400008 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 360
2026-08-11T22:25:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1595 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:58 dist-test-slave-5bn2 krb5kdc[537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:58 dist-test-slave-5bn2 krb5kdc[537](info): set up 2 sockets
Aug 11 22:25:58 dist-test-slave-5bn2 krb5kdc[537](info): commencing operation
krb5kdc: starting...
W20260811 22:25:59.602411 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:25:59 dist-test-slave-5bn2 krb5kdc[537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:25:59.737432 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:37015
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:37015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:59.853906 559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:59.854218 559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:59.854281 559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:59.858358 559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:59.858448 559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:59.858474 559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:59.858494 559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:59.858515 559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:59.865031 559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:37015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:37015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:25:59.866449 559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:59.867810 559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:59.877099 569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:59.877100 566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:59.877192 571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:59.877624 559 server_base.cc:1034] running on GCE node
I20260811 22:25:59.878224 559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:59.879428 559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:59.882725 559 hybrid_clock.cc:648] HybridClock initialized: now 1786487159882682 us; error 49 us; skew 500 ppm
Aug 11 22:25:59 dist-test-slave-5bn2 krb5kdc[537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487159, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:59.891796 559 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:25:59.892881 559 webserver.cc:460] Webserver started at http://127.26.125.62:38295/ using document root <none> and password file <none>
I20260811 22:25:59.893527 559 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:59.893601 559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:59.893832 559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:59.896209 559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "3e178fe5773045debc4d3dd1bd1b8579"
format_stamp: "Formatted at 2026-08-11 22:25:59 on dist-test-slave-5bn2"
server_key: "817b0022cb18dfab396794fd1ed2c221"
server_key_iv: "b62856201d3788f8435bb9ef03cea7b9"
server_key_version: "encryptionkey@0"
I20260811 22:25:59.896853 559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "3e178fe5773045debc4d3dd1bd1b8579"
format_stamp: "Formatted at 2026-08-11 22:25:59 on dist-test-slave-5bn2"
server_key: "817b0022cb18dfab396794fd1ed2c221"
server_key_iv: "b62856201d3788f8435bb9ef03cea7b9"
server_key_version: "encryptionkey@0"
I20260811 22:25:59.901103 559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:25:59.904232 583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:59.905642 559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:25:59.905768 559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "3e178fe5773045debc4d3dd1bd1b8579"
format_stamp: "Formatted at 2026-08-11 22:25:59 on dist-test-slave-5bn2"
server_key: "817b0022cb18dfab396794fd1ed2c221"
server_key_iv: "b62856201d3788f8435bb9ef03cea7b9"
server_key_version: "encryptionkey@0"
I20260811 22:25:59.905894 559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:59.926882 559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:59.930166 559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:59.930349 559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:59.937784 559 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:37015
I20260811 22:25:59.937798 666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:37015 every 8 connection(s)
I20260811 22:25:59.938799 559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:59.941761 667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:59.945433 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 559
I20260811 22:25:59.945574 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:25:59.945830 27124 external_mini_cluster.cc:1428] Setting key ab512a08e132f581134dbed734f8e80b
I20260811 22:25:59.947798 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579: Bootstrap starting.
I20260811 22:25:59.950311 667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:59.951226 667 log.cc:824] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:59.953554 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579: No bootstrap required, opened a new log
I20260811 22:25:59.955773 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.947372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47849 (local address 127.26.125.62:37015)
0811 22:25:59.947800 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:59.947817 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:59.947853 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:25:59.948521 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:59.948540 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:59.948902 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:59.949198 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:59.949564 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:59.950613 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:59.951834 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:59.952408 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:59.952539 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:59.953450 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:59.953462 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:59.953478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:59.953951 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:59.954025 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:59.954061 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:59.954150 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:59.954346 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 22:25:59.954581 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":118,"threads_started":1}
I20260811 22:25:59.956775 667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e178fe5773045debc4d3dd1bd1b8579" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37015 } }
I20260811 22:25:59.957007 667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:59.957057 667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e178fe5773045debc4d3dd1bd1b8579, State: Initialized, Role: FOLLOWER
I20260811 22:25:59.957463 667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e178fe5773045debc4d3dd1bd1b8579" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37015 } }
I20260811 22:25:59.957595 667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:59.957693 667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:59.957792 667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:59.958636 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:59.958670 667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e178fe5773045debc4d3dd1bd1b8579" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37015 } }
I20260811 22:25:59.959018 667 leader_election.cc:302] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e178fe5773045debc4d3dd1bd1b8579; no voters:
I20260811 22:25:59.959309 667 leader_election.cc:288] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:59.959416 673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:59.959591 673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [term 1 LEADER]: Becoming Leader. State: Replica: 3e178fe5773045debc4d3dd1bd1b8579, State: Running, Role: LEADER
I20260811 22:25:59.959965 673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e178fe5773045debc4d3dd1bd1b8579" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37015 } }
I20260811 22:25:59.960651 667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:59.961302 673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e178fe5773045debc4d3dd1bd1b8579. Latest consensus state: current_term: 1 leader_uuid: "3e178fe5773045debc4d3dd1bd1b8579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e178fe5773045debc4d3dd1bd1b8579" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37015 } } }
I20260811 22:25:59.961486 673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:59.961721 675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e178fe5773045debc4d3dd1bd1b8579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e178fe5773045debc4d3dd1bd1b8579" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 37015 } } }
I20260811 22:25:59.961840 675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:59.961901 693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:59.965456 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.960718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40423 (local address 127.26.125.62:37015)
0811 22:25:59.960826 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:59.960830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:59.961137 (+ 307us) server_negotiation.cc:407] Connection header received
0811 22:25:59.961325 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:59.961328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:59.961375 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:59.961427 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:59.961856 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:59.962570 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:59.963714 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:59.964035 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:59.964112 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:59.965013 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:59.965017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:59.965019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:59.965032 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:59.965060 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:59.965071 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:59.965130 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:59.965228 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 22:25:59.965278 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:25:59.965310 693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:59.972215 693 catalog_manager.cc:1347] Generated new cluster ID: bdb86cd7b2ef4f6f999809931e2478d3
I20260811 22:25:59.972282 693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:59.994447 693 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:59.995579 693 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:00.006386 693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e178fe5773045debc4d3dd1bd1b8579: Generated new TSK 0
I20260811 22:26:00.007498 693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:26:00.015208 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 559
2026-08-11T22:26:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1611 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:26:00 dist-test-slave-5bn2 krb5kdc[713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:26:00 dist-test-slave-5bn2 krb5kdc[713](info): set up 2 sockets
Aug 11 22:26:00 dist-test-slave-5bn2 krb5kdc[713](info): commencing operation
krb5kdc: starting...
W20260811 22:26:01.195399 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:26:01 dist-test-slave-5bn2 krb5kdc[713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:26:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:26:01.332448 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36421
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:33167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:36421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:01.450830 738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:01.451169 738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:01.451234 738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:01.455818 738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:01.455899 738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:01.455962 738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:01.455998 738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:01.456028 738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:01.461598 738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:33167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:36421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:36421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:26:01.462811 738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:01.463976 738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:01.472981 750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:01.473153 738 server_base.cc:1034] running on GCE node
W20260811 22:26:01.472965 754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:01.473342 747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:01.473973 738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:01.475203 738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:01.476377 738 hybrid_clock.cc:648] HybridClock initialized: now 1786487161476350 us; error 40 us; skew 500 ppm
Aug 11 22:26:01 dist-test-slave-5bn2 krb5kdc[713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487161, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:01.479197 738 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:26:01.480156 738 webserver.cc:460] Webserver started at http://127.26.125.62:35597/ using document root <none> and password file <none>
I20260811 22:26:01.480760 738 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:01.480829 738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:01.481037 738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:01.483273 738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "23161736f47547058974c884e6db579d"
format_stamp: "Formatted at 2026-08-11 22:26:01 on dist-test-slave-5bn2"
server_key: "633c53f9ba20e1db0cd8bab9a2f8f26c"
server_key_iv: "9f9e13ec6f4c9c54c858df3a99ee3a31"
server_key_version: "encryptionkey@0"
I20260811 22:26:01.483847 738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "23161736f47547058974c884e6db579d"
format_stamp: "Formatted at 2026-08-11 22:26:01 on dist-test-slave-5bn2"
server_key: "633c53f9ba20e1db0cd8bab9a2f8f26c"
server_key_iv: "9f9e13ec6f4c9c54c858df3a99ee3a31"
server_key_version: "encryptionkey@0"
I20260811 22:26:01.487933 738 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:26:01.490612 764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:01.492084 738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:26:01.492228 738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "23161736f47547058974c884e6db579d"
format_stamp: "Formatted at 2026-08-11 22:26:01 on dist-test-slave-5bn2"
server_key: "633c53f9ba20e1db0cd8bab9a2f8f26c"
server_key_iv: "9f9e13ec6f4c9c54c858df3a99ee3a31"
server_key_version: "encryptionkey@0"
I20260811 22:26:01.492349 738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:01.497257 738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:01.500610 738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:01.500805 738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:01.508036 738 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:36421
I20260811 22:26:01.508132 874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:36421 every 8 connection(s)
I20260811 22:26:01.509234 738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:01.510285 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 738
I20260811 22:26:01.510445 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:26:01.510780 27124 external_mini_cluster.cc:1428] Setting key 491679d3900acbf126f2909388d2d846
I20260811 22:26:01.512895 877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:01.516232 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.512343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49293 (local address 127.26.125.62:36421)
0811 22:26:01.512827 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:01.512844 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:01.512881 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:26:01.513577 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:01.513594 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:01.513885 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:01.514219 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:01.515041 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:01.515045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:01.515060 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:01.515070 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:01.515127 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:01.515146 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:01.515273 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:01.515353 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 22:26:01.515511 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":153,"threads_started":1}
I20260811 22:26:01.519004 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:26:01.519074 877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d: Bootstrap starting.
I20260811 22:26:01.521504 877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:01.522315 877 log.cc:824] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:01.522519 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.520943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49325 (local address 127.26.125.62:36421)
0811 22:26:01.521050 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:01.521054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:01.521232 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:26:01.521339 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:01.521342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:01.521373 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:01.521436 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:01.522159 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:01.522163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:01.522166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:01.522177 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:01.522205 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:01.522215 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:01.522284 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:01.522357 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 22:26:01.522393 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:26:01.524407 877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d: No bootstrap required, opened a new log
I20260811 22:26:01.528066 877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23161736f47547058974c884e6db579d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36421 } }
I20260811 22:26:01.528306 877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:01.528367 877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23161736f47547058974c884e6db579d, State: Initialized, Role: FOLLOWER
I20260811 22:26:01.528788 877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23161736f47547058974c884e6db579d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36421 } }
I20260811 22:26:01.528934 877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:26:01.529002 877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:26:01.529111 877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:01.530052 877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23161736f47547058974c884e6db579d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36421 } }
I20260811 22:26:01.530439 877 leader_election.cc:302] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23161736f47547058974c884e6db579d; no voters:
I20260811 22:26:01.530774 877 leader_election.cc:288] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:26:01.530912 897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:01.531162 897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [term 1 LEADER]: Becoming Leader. State: Replica: 23161736f47547058974c884e6db579d, State: Running, Role: LEADER
I20260811 22:26:01.531503 897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23161736f47547058974c884e6db579d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36421 } }
I20260811 22:26:01.532018 877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:01.533175 898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23161736f47547058974c884e6db579d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23161736f47547058974c884e6db579d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36421 } } }
I20260811 22:26:01.533310 898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:01.534018 899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23161736f47547058974c884e6db579d. Latest consensus state: current_term: 1 leader_uuid: "23161736f47547058974c884e6db579d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23161736f47547058974c884e6db579d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 36421 } } }
I20260811 22:26:01.534116 899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:01.534446 912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:01.537943 912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:01.543582 912 catalog_manager.cc:1347] Generated new cluster ID: 4fe548937b3e426d9055ad775d56c7c5
I20260811 22:26:01.543661 912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:01.578389 912 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:26:01.578749 912 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:01.582610 912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23161736f47547058974c884e6db579d: Generated new TSK 0
I20260811 22:26:01.583396 912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:26:01.588147 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 738
2026-08-11T22:26:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1563 ms)
[----------] 18 tests from AuthTokenIssuingTest (28751 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:26:01 dist-test-slave-5bn2 krb5kdc[937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:26:01 dist-test-slave-5bn2 krb5kdc[937](info): set up 2 sockets
Aug 11 22:26:01 dist-test-slave-5bn2 krb5kdc[937](info): commencing operation
krb5kdc: starting...
W20260811 22:26:02.792738 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:26:02 dist-test-slave-5bn2 krb5kdc[937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:26:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:26:02.928370 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40071
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40071,127.26.125.61:42489,127.26.125.60:32807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:03.045199 957 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:26:03.045550 957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:03.045575 957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:03.045636 957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:03.049845 957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:03.049932 957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:03.049949 957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:03.049965 957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:03.049979 957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:03.055678 957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40071,127.26.125.61:42489,127.26.125.60:32807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:40071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:26:03.056875 957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:03.057984 957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:03.066751 969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:03.066722 971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:03.066715 968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:03.067080 957 server_base.cc:1034] running on GCE node
I20260811 22:26:03.067719 957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:03.068889 957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:03.070071 957 hybrid_clock.cc:648] HybridClock initialized: now 1786487163070041 us; error 41 us; skew 500 ppm
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:03.073065 957 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:26:03.074103 957 webserver.cc:460] Webserver started at http://127.26.125.62:41985/ using document root <none> and password file <none>
I20260811 22:26:03.074674 957 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:03.074738 957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:03.074973 957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:03.077224 957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "2a533376eb2d403abcf02ef6addeeb7e"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "f38c0ece2f56d6554473c8098a2c692b"
server_key_iv: "80c32948a09c3f12196fbded9197b045"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.077818 957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "2a533376eb2d403abcf02ef6addeeb7e"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "f38c0ece2f56d6554473c8098a2c692b"
server_key_iv: "80c32948a09c3f12196fbded9197b045"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.082028 957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:26:03.084939 989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:03.086462 957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:03.086596 957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "2a533376eb2d403abcf02ef6addeeb7e"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "f38c0ece2f56d6554473c8098a2c692b"
server_key_iv: "80c32948a09c3f12196fbded9197b045"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.086727 957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:03.102242 957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:03.105795 957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:03.106004 957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:03.113281 957 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:40071
I20260811 22:26:03.113579 1081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:40071 every 8 connection(s)
I20260811 22:26:03.114333 957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:03.116792 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 957
I20260811 22:26:03.116945 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:26:03.117233 27124 external_mini_cluster.cc:1428] Setting key d9a624e4057cfc7f6e59e223a0064301
I20260811 22:26:03.117347 1084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:03.122123 1084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.125.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.61@KRBTEST.COM" created.
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:26:03.139438 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.125983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55862 (local address 127.26.125.62:40071)
0811 22:26:03.126327 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.126338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.126390 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:26:03.126998 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.127004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.127275 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.127453 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.128401 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.129406 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.130576 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.131028 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.131206 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.133967 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.133975 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.133992 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.134018 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.136094 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.136581 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.136588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.136589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.136747 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.137082 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.137085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.137087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.137694 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.137841 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.138263 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.138618 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260811 22:26:03.139444 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.125020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40071 (local address 127.0.0.1:55862)
0811 22:26:03.125622 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.125677 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.126622 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.127528 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.127534 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.127883 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.128206 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.128226 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.129547 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.129550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.130408 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.130418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.131213 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.131217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.131325 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.131956 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.131980 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.133756 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.136203 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.136213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.136223 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.136468 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.136850 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.136853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.136854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.136990 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.137838 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.137844 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.137993 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.138147 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.138489 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":129,"threads_started":1}
I20260811 22:26:03.142462 1084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:03.143289 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.143009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52368)
0811 22:26:03.143141 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.143218 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:26:03.143564 996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
I20260811 22:26:03.144483 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.144255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52380)
0811 22:26:03.144369 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.144417 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:26:03.144608 1084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:42489: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab.
Entry for principal kudu/127.26.125.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab.
I20260811 22:26:03.149842 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.61:42489
--webserver_interface=127.26.125.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab
--principal=kudu/127.26.125.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40071,127.26.125.61:42489,127.26.125.60:32807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:26:03.193873 1084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } attempt: 1
I20260811 22:26:03.194734 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.194437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52386)
0811 22:26:03.194590 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.194655 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:26:03.195652 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.195418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52400)
0811 22:26:03.195533 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.195586 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:26:03.195781 1084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:42489: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
I20260811 22:26:03.263964 1084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } attempt: 2
I20260811 22:26:03.264864 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.264536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52406)
0811 22:26:03.264702 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.264773 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 22:26:03.265767 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.265526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52420)
0811 22:26:03.265642 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.265698 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:26:03.265905 1084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:42489: Network error: Client connection negotiation failed: client connection to 127.26.125.61:42489: connect: Connection refused (error 111)
W20260811 22:26:03.268242 1096 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:26:03.268594 1096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:03.268639 1096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:03.268704 1096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:03.273058 1096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:03.273142 1096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:03.273169 1096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:03.273191 1096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:03.273221 1096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:03.279150 1096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40071,127.26.125.61:42489,127.26.125.60:32807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab
--principal=kudu/127.26.125.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.61:42489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.125.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:26:03.280411 1096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:03.281559 1096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:03.290983 1112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:03.290973 1101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:03.290973 1102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:03.291056 1096 server_base.cc:1034] running on GCE node
I20260811 22:26:03.291837 1096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:03.293121 1096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:03.294301 1096 hybrid_clock.cc:648] HybridClock initialized: now 1786487163294276 us; error 43 us; skew 500 ppm
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:03.297344 1096 init.cc:377] Logged in from keytab as kudu/127.26.125.61@KRBTEST.COM (short username kudu)
I20260811 22:26:03.298389 1096 webserver.cc:460] Webserver started at http://127.26.125.61:35381/ using document root <none> and password file <none>
I20260811 22:26:03.299036 1096 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:03.299114 1096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:03.299341 1096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:03.301643 1096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/data/instance:
uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "3355bcfa93e78dd2670ace7fcb3e3bcc"
server_key_iv: "5ed0929ccecf510d5a8f02d283635c0c"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.302294 1096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/wal/instance:
uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "3355bcfa93e78dd2670ace7fcb3e3bcc"
server_key_iv: "5ed0929ccecf510d5a8f02d283635c0c"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.306694 1096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 22:26:03.309501 1121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:03.311064 1096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:03.311206 1096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/wal
uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "3355bcfa93e78dd2670ace7fcb3e3bcc"
server_key_iv: "5ed0929ccecf510d5a8f02d283635c0c"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.311333 1096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:03.334167 1096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:03.337726 1096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:03.337917 1096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:03.345427 1096 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.61:42489
I20260811 22:26:03.345742 1198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.61:42489 every 8 connection(s)
I20260811 22:26:03.346503 1096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
I20260811 22:26:03.347896 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1096
I20260811 22:26:03.348030 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-1/wal/instance
I20260811 22:26:03.348315 27124 external_mini_cluster.cc:1428] Setting key 197f96d0b9cda7f84d20e455e11411e6
I20260811 22:26:03.349714 1199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:03.353802 1199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.125.60@KRBTEST.COM; defaulting to no policy
I20260811 22:26:03.357055 1084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.60@KRBTEST.COM" created.
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:26:03.368892 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.357609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52424)
0811 22:26:03.357798 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.357812 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.357899 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.359144 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.359147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.359360 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.359590 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.359601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.361024 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.361028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.361795 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.361804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.362477 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.362481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.362584 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.363150 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.363164 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.364656 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.366795 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.366799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.366801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.367165 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.367566 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.367569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.367571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.367667 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.368341 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.368344 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.368464 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.368557 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.368716 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 22:26:03.370283 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.357726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.26.125.61:42489)
0811 22:26:03.358066 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.358077 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.358121 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:26:03.358643 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.358674 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.358964 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.359156 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.359755 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.360877 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.361940 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.362344 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.362501 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.364804 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.364812 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.364829 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.364854 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.366667 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.367277 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.367284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.367290 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.367439 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.367759 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.367763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.367764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.368234 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.368366 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.368746 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.369101 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":241,"thread_start_us":138,"threads_started":1}
I20260811 22:26:03.371177 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.356527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40071 (local address 127.0.0.1:55872)
0811 22:26:03.357485 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.357554 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.358628 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.359110 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.359117 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.359492 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.359855 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.359867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.360672 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.360675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.361783 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.361795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.363083 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.363088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.363214 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.364502 (+ 1288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.364522 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.366231 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.369231 (+ 3000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.369239 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.369247 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.369526 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.369847 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.369850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.369852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.369946 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.370528 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.370535 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.370668 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.370788 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.371007 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":4,"thread_start_us":152,"threads_started":1}
I20260811 22:26:03.371289 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.355954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.26.125.62:40071)
0811 22:26:03.356084 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.356088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.357670 (+ 1582us) server_negotiation.cc:407] Connection header received
0811 22:26:03.358889 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.358893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.358955 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.359028 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.359998 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.360535 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.361921 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.362839 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.362992 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.366401 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.366406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.366409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.366448 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.368226 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.369637 (+ 1411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.369641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.369643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.369738 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.370039 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.370042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.370044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.370293 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.370909 (+ 616us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.371025 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.371148 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:26:03.372860 1084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:03.373008 1199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:03.373697 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.373358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:36994)
0811 22:26:03.373550 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.373612 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 22:26:03.374490 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.374196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:36998)
0811 22:26:03.374348 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.374406 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 22:26:03.374617 1084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:32807: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab.
Entry for principal kudu/127.26.125.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab.
I20260811 22:26:03.381729 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.373493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52438)
0811 22:26:03.373636 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.373648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.373731 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.374059 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.374061 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.374225 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.374342 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.374350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.375328 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.375331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.375927 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.375937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.376514 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.376518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.376602 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.377108 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.377120 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.378473 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.380149 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.380152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.380154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.380371 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.380666 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.380669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.380671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.380766 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.381172 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.381175 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.381272 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.381381 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.381546 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 22:26:03.381734 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.373577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.26.125.61:42489)
0811 22:26:03.373694 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.373699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.373714 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:03.373823 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.373826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.373866 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.373910 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.374558 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.375228 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.376079 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.376383 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.376471 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.378621 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.378624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.378626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.378650 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.380031 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.380487 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.380490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.380491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.380579 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.380859 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.380862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.380864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.381082 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.381179 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.381509 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.381614 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:26:03.382016 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.60:32807
--webserver_interface=127.26.125.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab
--principal=kudu/127.26.125.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:37151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:40071,127.26.125.61:42489,127.26.125.60:32807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:26:03.382540 1199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:03.383327 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.383025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37006)
0811 22:26:03.383152 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.383254 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:26:03.383584 1125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
I20260811 22:26:03.384534 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.384315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37012)
0811 22:26:03.384430 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.384481 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:26:03.384662 1199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:32807: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
I20260811 22:26:03.405799 1084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } attempt: 1
I20260811 22:26:03.406793 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.406445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37028)
0811 22:26:03.406640 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.406708 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:26:03.407685 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.407417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37042)
0811 22:26:03.407570 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.407621 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 22:26:03.407809 1084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:32807: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
I20260811 22:26:03.433966 1199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } attempt: 1
I20260811 22:26:03.434955 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.434588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37056)
0811 22:26:03.434765 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.434838 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:26:03.436236 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.436013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37060)
0811 22:26:03.436123 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.436173 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:26:03.436359 1199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:32807: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
I20260811 22:26:03.482999 1084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } attempt: 2
I20260811 22:26:03.483940 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.483623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37066)
0811 22:26:03.483783 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.483851 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 22:26:03.484743 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.484528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37076)
0811 22:26:03.484630 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.484681 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 22:26:03.484886 1084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:32807: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
W20260811 22:26:03.502094 1212 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:26:03.502418 1212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:03.502441 1212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:03.502512 1212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:26:03.504628 1199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } attempt: 2
I20260811 22:26:03.505532 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.505194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37082)
0811 22:26:03.505341 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.505408 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 22:26:03.506300 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.506077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37096)
0811 22:26:03.506186 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.506239 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:26:03.506474 1199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:32807: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
W20260811 22:26:03.506980 1212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:03.507040 1212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:03.507059 1212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:03.507081 1212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:03.507099 1212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:03.512794 1212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:37151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:40071,127.26.125.61:42489,127.26.125.60:32807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab
--principal=kudu/127.26.125.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.60:32807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.125.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:26:03.513978 1212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:03.515157 1212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:03.524089 1221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:03.524080 1223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:03.524151 1212 server_base.cc:1034] running on GCE node
W20260811 22:26:03.524080 1220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:03.524964 1212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:03.526243 1212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:03.527444 1212 hybrid_clock.cc:648] HybridClock initialized: now 1786487163527423 us; error 37 us; skew 500 ppm
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:03.530701 1212 init.cc:377] Logged in from keytab as kudu/127.26.125.60@KRBTEST.COM (short username kudu)
I20260811 22:26:03.531798 1212 webserver.cc:460] Webserver started at http://127.26.125.60:46219/ using document root <none> and password file <none>
I20260811 22:26:03.532404 1212 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:03.532487 1212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:03.532716 1212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:03.535109 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/data/instance:
uuid: "38bf47fc44274bbaa3afc221a12af575"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "137dad6457c4b1901148bad424442002"
server_key_iv: "80ec9fa32a00c190e7e7a55ce8487d10"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.535815 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/wal/instance:
uuid: "38bf47fc44274bbaa3afc221a12af575"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "137dad6457c4b1901148bad424442002"
server_key_iv: "80ec9fa32a00c190e7e7a55ce8487d10"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.540031 1212 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:26:03.542729 1231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:03.544113 1212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:03.544246 1212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/wal
uuid: "38bf47fc44274bbaa3afc221a12af575"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-5bn2"
server_key: "137dad6457c4b1901148bad424442002"
server_key_iv: "80ec9fa32a00c190e7e7a55ce8487d10"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.544358 1212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:03.584101 1084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } attempt: 3
I20260811 22:26:03.585076 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.584697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37110)
0811 22:26:03.584862 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.584941 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:26:03.585922 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.585692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37120)
0811 22:26:03.585795 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.585852 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:26:03.586103 1084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:32807: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
I20260811 22:26:03.597661 1199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } attempt: 3
I20260811 22:26:03.598657 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.598276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37136)
0811 22:26:03.598480 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.598561 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 22:26:03.599587 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.599294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37146)
0811 22:26:03.599463 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.599521 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 22:26:03.599733 1199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:32807: Network error: Client connection negotiation failed: client connection to 127.26.125.60:32807: connect: Connection refused (error 111)
I20260811 22:26:03.605707 1212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:03.609385 1212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:03.609575 1212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:03.617110 1212 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.60:32807
I20260811 22:26:03.617479 1329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.60:32807 every 8 connection(s)
I20260811 22:26:03.618186 1212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
I20260811 22:26:03.621008 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1212
I20260811 22:26:03.621129 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486868287078-27124-0/minicluster-data/master-2/wal/instance
I20260811 22:26:03.621284 1331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:03.621423 27124 external_mini_cluster.cc:1428] Setting key 3957874e7dee9bba3b6290fe0e6e0a28
I20260811 22:26:03.626353 1331 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:26:03.637550 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.622780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.26.125.60:32807)
0811 22:26:03.623223 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.623239 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.623276 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:26:03.624071 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.624090 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.624511 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.624808 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.625849 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.626912 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.628131 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.628917 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.629077 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.632084 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.632091 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.632104 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.632141 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.634031 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.634550 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.634557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.634562 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.634683 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.635034 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.635037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.635038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.635610 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.635817 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.636252 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.636596 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":122,"threads_started":1}
I20260811 22:26:03.638401 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.628501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.26.125.62:40071)
0811 22:26:03.628627 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.628631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.629081 (+ 450us) server_negotiation.cc:407] Connection header received
0811 22:26:03.629344 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.629347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.629400 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.629475 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.630433 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.631006 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.631967 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.632282 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.632395 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.634887 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.634892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.634894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.634926 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.636518 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.637020 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.637024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.637026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.637115 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.637364 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.637368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.637370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.637567 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.637652 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.638166 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.638277 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:26:03.638433 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.628508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:40071 (local address 127.0.0.1:55878)
0811 22:26:03.628931 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.628985 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.629188 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.629554 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.629560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.630094 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.630284 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.630292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.631130 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.631133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.631838 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.631850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.632415 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.632418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.632596 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.633096 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.633112 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.634696 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.636622 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.636631 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.636638 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.636922 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.637199 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.637203 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.637205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.637285 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.637788 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.637795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.637932 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.638080 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.638261 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":158,"threads_started":1}
I20260811 22:26:03.639839 1331 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:26:03.648741 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.640606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.26.125.61:42489)
0811 22:26:03.640748 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.640752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.640766 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:03.640805 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.640808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.640850 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.640931 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.641357 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.641937 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.642735 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.643084 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.643176 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.645302 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.645305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.645307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.645336 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.646840 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.647351 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.647353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.647355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.647441 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.647759 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.647763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.647765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.648005 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.648105 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.648453 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.648575 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:26:03.648762 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.640320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:42489 (local address 127.0.0.1:52442)
0811 22:26:03.640578 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.640592 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.640666 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.640954 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.640956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.641120 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.641240 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.641247 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.642049 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.642053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.642614 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.642624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.643194 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.643197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.643296 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.643768 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.643780 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.645133 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.646984 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.646989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.646991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.647255 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.647560 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.647564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.647566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.647658 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.648153 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.648156 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.648266 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.648361 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.648532 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 22:26:03.649540 1331 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:03.649750 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.640320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55890 (local address 127.26.125.62:40071)
0811 22:26:03.640434 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.640438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.640453 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:03.640569 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.640572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.640617 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.640680 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.641536 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.642029 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.642834 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.643101 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.643198 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.645998 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.646002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.646004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.646039 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.647634 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.648184 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.648188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.648191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.648282 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.648546 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.648550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.648552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.648759 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.649079 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.649473 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.649605 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
I20260811 22:26:03.658404 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.650169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37154)
0811 22:26:03.650303 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.650318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.650398 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.650668 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.650671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.650877 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.650982 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.650989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.652045 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.652048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.652742 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.652752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.653205 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.653208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.653271 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.653782 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.653794 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.655396 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.657031 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.657035 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.657037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.657217 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.657453 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.657463 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.657466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.657540 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.657883 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.657886 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.657974 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.658060 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.658210 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 22:26:03.658404 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.650310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.26.125.60:32807)
0811 22:26:03.650417 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.650420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.650433 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:26:03.650463 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.650465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.650552 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.650636 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.651103 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.651932 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.652855 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.653100 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.653198 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.655600 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.655604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.655607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.655636 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.656925 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.657300 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.657304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.657305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.657376 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.657615 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.657618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.657620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.657799 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.657882 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.658113 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.658262 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":39}
I20260811 22:26:03.660018 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.651225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.26.125.61:42489)
0811 22:26:03.651332 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.651336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.651406 (+ 70us) server_negotiation.cc:407] Connection header received
0811 22:26:03.651537 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.651539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.651575 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.651635 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.652355 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.653009 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.653960 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.654308 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.654404 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.657054 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.657059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.657061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.657088 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.658377 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.658820 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.658824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.658825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.658947 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.659211 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.659215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.659217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.659393 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.659646 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.659751 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.659864 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:26:03.662256 1331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575: Bootstrap starting.
I20260811 22:26:03.663805 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:26:03.665236 1331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:03.666261 1331 log.cc:824] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:03.665446 1039 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2a533376eb2d403abcf02ef6addeeb7e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55890
I20260811 22:26:03.669001 1331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575: No bootstrap required, opened a new log
I20260811 22:26:03.673276 1331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } }
I20260811 22:26:03.673592 1331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38bf47fc44274bbaa3afc221a12af575, State: Initialized, Role: FOLLOWER
I20260811 22:26:03.673964 1331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } }
I20260811 22:26:03.674533 1344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.674683 1344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:03.675510 1331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:26:03.680927 1365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:26:03.681028 1365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:26:03.679425 1212 master_runner.cc:186] Error getting master registration for 127.26.125.62:40071: OK, instance_id { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" instance_seqno: 1786487163111323 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:26:03.682293 1212 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:26:03.742887 1199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } attempt: 4
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
I20260811 22:26:03.750218 1084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } attempt: 4
I20260811 22:26:03.753000 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.743542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37160)
0811 22:26:03.743704 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.743722 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.743840 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.744089 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.744092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.744327 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.744453 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.744460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.745202 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.745206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.746070 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.746080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.746718 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.746722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.746811 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.747412 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.747425 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.749065 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.751321 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.751325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.751327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.751584 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.752039 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.752043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.752045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.752150 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.752531 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.752534 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.752641 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.752745 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.752872 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:26:03.754182 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.743640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.26.125.60:32807)
0811 22:26:03.743763 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.743767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.743783 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:26:03.743931 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.743934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.743984 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.744055 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.744576 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.745085 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.746222 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.746587 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.746716 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.749219 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.749223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.749226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.749256 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.751207 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.751842 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.751846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.751848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.751948 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.752237 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.752240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.752242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.752448 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.752541 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.753896 (+ 1355us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.754029 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 22:26:03 dist-test-slave-5bn2 krb5kdc[937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
I20260811 22:26:03.757716 1199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0: Bootstrap starting.
I20260811 22:26:03.760036 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.750728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:32807 (local address 127.0.0.1:37172)
0811 22:26:03.751007 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.751024 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.751113 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.751402 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.751405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.751592 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.751704 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.751711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.752723 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.752727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.753559 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.753571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.754166 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.754170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.754314 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.754891 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.754911 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.756445 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.758393 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.758396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.758398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.758645 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.758987 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.758990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.758992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.759089 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.759512 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.759514 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.759619 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.759720 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.759858 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260811 22:26:03.760212 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.750762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.26.125.60:32807)
0811 22:26:03.751177 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.751188 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.751204 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:26:03.751240 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.751243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.751288 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.751377 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.751815 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.752610 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.753693 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.754054 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.754145 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.756572 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.756575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.756578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.756603 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.758277 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.758765 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.758769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.758771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.758896 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.759183 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.759186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.759188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.759417 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.759517 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.759789 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.759921 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":145,"threads_started":1}
I20260811 22:26:03.760927 1199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:03.761899 1199 log.cc:824] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:03.763568 1084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e: Bootstrap starting.
I20260811 22:26:03.764453 1199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0: No bootstrap required, opened a new log
I20260811 22:26:03.766384 1084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:03.767289 1084 log.cc:824] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:03.768442 1199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } }
I20260811 22:26:03.768635 1199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f89d40bd9fc4df6833a9ad6cf6dccc0, State: Initialized, Role: FOLLOWER
I20260811 22:26:03.768950 1199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } }
I20260811 22:26:03.769479 1373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.769670 1373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:03.770533 1199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:03.775058 1084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e: No bootstrap required, opened a new log
W20260811 22:26:03.776312 1390 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:26:03.776386 1390 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:26:03.774384 1096 master_runner.cc:186] Error getting master registration for 127.26.125.62:40071: OK, instance_id { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" instance_seqno: 1786487163111323 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:26:03.777747 1096 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:26:03.778326 1084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } }
I20260811 22:26:03.778594 1084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a533376eb2d403abcf02ef6addeeb7e, State: Initialized, Role: FOLLOWER
I20260811 22:26:03.779089 1084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } }
I20260811 22:26:03.779685 1394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.779883 1394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:03.780645 1084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:03.780736 1039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:26:03.780833 1039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:03.782249 1039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } }
I20260811 22:26:03.783339 1039 leader_election.cc:288] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [CANDIDATE]: Term 1 election: Requested vote from peers 5f89d40bd9fc4df6833a9ad6cf6dccc0 (127.26.125.61:42489), 38bf47fc44274bbaa3afc221a12af575 (127.26.125.60:32807)
I20260811 22:26:03.784511 1277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a533376eb2d403abcf02ef6addeeb7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "38bf47fc44274bbaa3afc221a12af575"
I20260811 22:26:03.784516 1157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a533376eb2d403abcf02ef6addeeb7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0"
I20260811 22:26:03.784775 1157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:03.784777 1277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:03.786209 1277 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a533376eb2d403abcf02ef6addeeb7e in term 1.
I20260811 22:26:03.786213 1157 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a533376eb2d403abcf02ef6addeeb7e in term 1.
I20260811 22:26:03.786893 996 leader_election.cc:302] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a533376eb2d403abcf02ef6addeeb7e, 38bf47fc44274bbaa3afc221a12af575; no voters:
I20260811 22:26:03.787217 1394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:03.787425 1394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [term 1 LEADER]: Becoming Leader. State: Replica: 2a533376eb2d403abcf02ef6addeeb7e, State: Running, Role: LEADER
I20260811 22:26:03.787776 1394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } }
W20260811 22:26:03.788693 1412 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:26:03.788764 1412 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:26:03.789705 1394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a533376eb2d403abcf02ef6addeeb7e. Latest consensus state: current_term: 1 leader_uuid: "2a533376eb2d403abcf02ef6addeeb7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.789813 1394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:03.790249 1434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:03.791303 1434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:03.791885 957 master_runner.cc:263] No leader master found from master 2a533376eb2d403abcf02ef6addeeb7e
I20260811 22:26:03.791966 957 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:26:03.803483 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.785761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.26.125.61:42489)
0811 22:26:03.785881 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.785885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.793659 (+ 7774us) server_negotiation.cc:407] Connection header received
0811 22:26:03.793871 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.793874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.793928 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.793982 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.795018 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.795767 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.796939 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.797276 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.797366 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.799513 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.799517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.799520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.799549 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.801479 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.801978 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.801982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.801984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.802076 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.802401 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.802405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.802407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.802643 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.802768 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.803169 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.803298 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:26:03.803588 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.790212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.26.125.60:32807)
0811 22:26:03.790342 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.790346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.793863 (+ 3517us) server_negotiation.cc:407] Connection header received
0811 22:26:03.794078 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.794081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.794129 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.794179 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.795134 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.795892 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.797055 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.797415 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.797496 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.799706 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.799710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.799713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.799743 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.801642 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.802150 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.802154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.802156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.802247 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.802566 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.802570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.802572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.802797 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.802986 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.803315 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.803441 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:26:03.807576 1157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [term 1 FOLLOWER]: Refusing update from remote peer 2a533376eb2d403abcf02ef6addeeb7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:03.807576 1277 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [term 1 FOLLOWER]: Refusing update from remote peer 2a533376eb2d403abcf02ef6addeeb7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:03.808317 1394 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:03.808555 1424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:03.813580 1373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a533376eb2d403abcf02ef6addeeb7e. Latest consensus state: current_term: 1 leader_uuid: "2a533376eb2d403abcf02ef6addeeb7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.813746 1373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:03.818105 1344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a533376eb2d403abcf02ef6addeeb7e. Latest consensus state: current_term: 1 leader_uuid: "2a533376eb2d403abcf02ef6addeeb7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.818253 1344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:03.818471 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.786187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55900 (local address 127.26.125.62:40071)
0811 22:26:03.786294 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.786298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.803158 (+ 16860us) server_negotiation.cc:407] Connection header received
0811 22:26:03.808147 (+ 4989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.808151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.808210 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.808289 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.809213 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.809988 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.811339 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.811815 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.811934 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.813962 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.813967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.813970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.814008 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.815885 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.816547 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.816551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.816554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.816666 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.817002 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.817006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.817008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.817266 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.817403 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.818181 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.818306 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:26:03.823292 1394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2a533376eb2d403abcf02ef6addeeb7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.823446 1394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:03.823973 1373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2a533376eb2d403abcf02ef6addeeb7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.824102 1373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:03.824306 1435 mvcc.cc:205] Tried to move back new op lower bound from 7317451422948474880 to 7317451422878347264. Current Snapshot: MvccSnapshot[applied={T|T < 7317451422948474880}]
I20260811 22:26:03.824859 1437 mvcc.cc:205] Tried to move back new op lower bound from 7317451422948474880 to 7317451422878347264. Current Snapshot: MvccSnapshot[applied={T|T < 7317451422948474880}]
I20260811 22:26:03.830492 1424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2a533376eb2d403abcf02ef6addeeb7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.830644 1424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:03.832015 1344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2a533376eb2d403abcf02ef6addeeb7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a533376eb2d403abcf02ef6addeeb7e" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 40071 } } peers { permanent_uuid: "5f89d40bd9fc4df6833a9ad6cf6dccc0" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 42489 } } peers { permanent_uuid: "38bf47fc44274bbaa3afc221a12af575" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 32807 } } }
I20260811 22:26:03.832132 1344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:03.833968 1434 catalog_manager.cc:1347] Generated new cluster ID: e909644c8df74c87b04c66f8e758eea0
I20260811 22:26:03.834045 1434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:03.863925 1434 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:26:03.865307 1434 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:03.880040 1434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a533376eb2d403abcf02ef6addeeb7e: Generated new TSK 0
I20260811 22:26:03.881019 1434 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:26:03.920200 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.918340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.26.125.61:42489)
0811 22:26:03.918462 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.918466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.919090 (+ 624us) server_negotiation.cc:407] Connection header received
0811 22:26:03.919280 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.919284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.919332 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.919381 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.920057 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52456: BlockingRecv error: recv got EOF from 127.0.0.1:52456 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 22:26:03.920204 1092 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.917881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.26.125.62:40071)
0811 22:26:03.918012 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.918017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.919067 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 22:26:03.919290 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.919293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.919345 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.919398 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.920067 (+ 669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55916: BlockingRecv error: recv got EOF from 127.0.0.1:55916 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:26:03.921499 1368 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.918658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.26.125.60:32807)
0811 22:26:03.918773 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.918777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.920654 (+ 1877us) server_negotiation.cc:407] Connection header received
0811 22:26:03.920855 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.920858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.920913 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.920997 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.921384 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37190: BlockingRecv error: recv got EOF from 127.0.0.1:37190 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:26:03.924760 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.923123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52464 (local address 127.26.125.61:42489)
0811 22:26:03.923225 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.923228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.923835 (+ 607us) server_negotiation.cc:407] Connection header received
0811 22:26:03.924095 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.924098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.924178 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.924263 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.924654 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52464: BlockingRecv error: recv got EOF from 127.0.0.1:52464 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 22:26:03.928897 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.928088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.26.125.61:42489)
0811 22:26:03.928199 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.928203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.928215 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:26:03.928254 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.928257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.928301 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.928375 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.928796 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52468: BlockingRecv error: recv got EOF from 127.0.0.1:52468 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 22:26:03.936084 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.933100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52484 (local address 127.26.125.61:42489)
0811 22:26:03.933215 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.933219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.935215 (+ 1996us) server_negotiation.cc:407] Connection header received
0811 22:26:03.935349 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.935353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.935412 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.935501 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.935963 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52484: BlockingRecv error: recv got EOF from 127.0.0.1:52484 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 22:26:03.943404 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.942070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.26.125.61:42489)
0811 22:26:03.942194 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.942198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.942442 (+ 244us) server_negotiation.cc:407] Connection header received
0811 22:26:03.942601 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.942604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.942662 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.942758 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.943285 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52500: BlockingRecv error: recv got EOF from 127.0.0.1:52500 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 22:26:03.955282 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.953820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.26.125.61:42489)
0811 22:26:03.953960 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.953965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.954256 (+ 291us) server_negotiation.cc:407] Connection header received
0811 22:26:03.954439 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.954443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.954503 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.954587 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.955157 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52506: BlockingRecv error: recv got EOF from 127.0.0.1:52506 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:26:03.975159 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:03.973768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.26.125.61:42489)
0811 22:26:03.973922 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.973927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.974192 (+ 265us) server_negotiation.cc:407] Connection header received
0811 22:26:03.974331 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.974335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.974400 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.974506 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.975030 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52510: BlockingRecv error: recv got EOF from 127.0.0.1:52510 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 22:26:04.011008 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:04.009548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52520 (local address 127.26.125.61:42489)
0811 22:26:04.009765 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.009770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.010022 (+ 252us) server_negotiation.cc:407] Connection header received
0811 22:26:04.010196 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.010199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.010255 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.010334 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.010890 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52520: BlockingRecv error: recv got EOF from 127.0.0.1:52520 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260811 22:26:04.078758 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:04.077428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52528 (local address 127.26.125.61:42489)
0811 22:26:04.077614 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.077619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.077806 (+ 187us) server_negotiation.cc:407] Connection header received
0811 22:26:04.077963 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.077966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.078024 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.078105 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.078629 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52528: BlockingRecv error: recv got EOF from 127.0.0.1:52528 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 22:26:04.210198 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:04.209091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.26.125.61:42489)
0811 22:26:04.209243 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.209246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.209342 (+ 96us) server_negotiation.cc:407] Connection header received
0811 22:26:04.209475 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.209478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.209527 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.209608 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.210092 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52530: BlockingRecv error: recv got EOF from 127.0.0.1:52530 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:26:04.470151 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:04.468859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.26.125.61:42489)
0811 22:26:04.468988 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.468991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.469087 (+ 96us) server_negotiation.cc:407] Connection header received
0811 22:26:04.469266 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.469269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.469325 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.469416 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.469963 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52544: BlockingRecv error: recv got EOF from 127.0.0.1:52544 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:26:04.682580 1365 catalog_manager.cc:1259] Loaded cluster ID: e909644c8df74c87b04c66f8e758eea0
I20260811 22:26:04.682674 1365 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575: loading cluster ID for follower catalog manager: success
I20260811 22:26:04.684497 1365 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575: acquiring CA information for follower catalog manager: success
I20260811 22:26:04.685801 1365 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 38bf47fc44274bbaa3afc221a12af575: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:26:04.777835 1390 catalog_manager.cc:1259] Loaded cluster ID: e909644c8df74c87b04c66f8e758eea0
I20260811 22:26:04.777931 1390 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0: loading cluster ID for follower catalog manager: success
I20260811 22:26:04.779735 1390 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0: acquiring CA information for follower catalog manager: success
I20260811 22:26:04.781178 1390 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5f89d40bd9fc4df6833a9ad6cf6dccc0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:26:04.989162 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.984565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52554 (local address 127.26.125.61:42489)
0811 22:26:04.984879 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.984882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.984900 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:26:04.985008 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.985011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.985148 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.985270 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:04.985608 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.986188 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.987199 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.987527 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.987639 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.988072 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:04.988630 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:04.988742 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.988893 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.988970 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":101,"threads_started":1}
I20260811 22:26:04.997166 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.993055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.26.125.60:32807)
0811 22:26:04.993387 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.993398 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.993413 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:04.993550 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.993553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.993720 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.993836 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:04.994092 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.994660 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.995585 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.995824 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.995914 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.996237 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:04.996721 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:04.996840 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.996905 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.996976 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":70,"threads_started":1}
I20260811 22:26:05.004716 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.001206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.26.125.60:32807)
0811 22:26:05.001341 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.001345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.001359 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:05.001416 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.001419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.001467 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.001533 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:05.001771 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.002573 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.003716 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.003943 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.004033 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.004265 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.004341 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.004399 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.004500 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.004558 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:26:05.005486 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.000725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.26.125.61:42489)
0811 22:26:05.000852 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.000857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.001193 (+ 336us) server_negotiation.cc:407] Connection header received
0811 22:26:05.001446 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.001450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.001515 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.001582 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:05.001932 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.002693 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.004076 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.004433 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.004514 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.004829 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.004921 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.004997 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.005285 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.005350 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 22:26:05.006155 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.000639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.26.125.62:40071)
0811 22:26:05.001533 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.001538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.001550 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:26:05.001654 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.001657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.001843 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.001979 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:05.002194 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.002753 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.004076 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.004425 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.004818 (+ 393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.004980 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.005380 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.005957 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.005987 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.006042 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":95,"threads_started":1}
I20260811 22:26:05.012884 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 957
I20260811 22:26:05.031857 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1096
I20260811 22:26:05.045542 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1212
2026-08-11T22:26:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3464 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:26:05 dist-test-slave-5bn2 krb5kdc[1600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:26:05 dist-test-slave-5bn2 krb5kdc[1600](info): set up 2 sockets
Aug 11 22:26:05 dist-test-slave-5bn2 krb5kdc[1600](info): commencing operation
krb5kdc: starting...
W20260811 22:26:06.235841 27124 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/test-user.keytab.
Aug 11 22:26:06 dist-test-slave-5bn2 krb5kdc[1600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:26:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.125.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
Entry for principal kudu/127.26.125.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab.
I20260811 22:26:06.369586 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:39335
--webserver_interface=127.26.125.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:39335,127.26.125.61:43527,127.26.125.60:42605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:06.485819 1621 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:26:06.486171 1621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:06.486208 1621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:06.486276 1621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:06.490729 1621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:06.490825 1621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:06.490876 1621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:06.490901 1621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:06.490921 1621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:06.496454 1621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:39335,127.26.125.61:43527,127.26.125.60:42605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.62.keytab
--principal=kudu/127.26.125.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.62:39335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.125.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:26:06.497612 1621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:06.498710 1621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:06.507706 1634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:06.507942 1621 server_base.cc:1034] running on GCE node
W20260811 22:26:06.507711 1630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:06.507711 1631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:06.508697 1621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:06.509881 1621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:06.511081 1621 hybrid_clock.cc:648] HybridClock initialized: now 1786487166511048 us; error 57 us; skew 500 ppm
Aug 11 22:26:06 dist-test-slave-5bn2 krb5kdc[1600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:06.514221 1621 init.cc:377] Logged in from keytab as kudu/127.26.125.62@KRBTEST.COM (short username kudu)
I20260811 22:26:06.515340 1621 webserver.cc:460] Webserver started at http://127.26.125.62:44021/ using document root <none> and password file <none>
I20260811 22:26:06.515940 1621 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:06.516013 1621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:06.516223 1621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:06.518420 1621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/data/instance:
uuid: "52f324d607bc4043b2f82983af4a622d"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "c27d3faf6f9d152c0b3d786c0f2f186b"
server_key_iv: "10ea186f658a9e894f7046b03c73d2ee"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.519034 1621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/wal/instance:
uuid: "52f324d607bc4043b2f82983af4a622d"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "c27d3faf6f9d152c0b3d786c0f2f186b"
server_key_iv: "10ea186f658a9e894f7046b03c73d2ee"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.523123 1621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:26:06.525695 1654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:06.527127 1621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:06.527264 1621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/wal
uuid: "52f324d607bc4043b2f82983af4a622d"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "c27d3faf6f9d152c0b3d786c0f2f186b"
server_key_iv: "10ea186f658a9e894f7046b03c73d2ee"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.527379 1621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:06.549362 1621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:06.552871 1621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:06.553076 1621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:06.560459 1621 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.62:39335
I20260811 22:26:06.560724 1750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.62:39335 every 8 connection(s)
I20260811 22:26:06.561489 1621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:06.564579 1751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:06.567576 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1621
I20260811 22:26:06.567708 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-0/wal/instance
I20260811 22:26:06.568002 27124 external_mini_cluster.cc:1428] Setting key e857158545b73f062117524625053241
I20260811 22:26:06.568763 1751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.125.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.61@KRBTEST.COM" created.
Aug 11 22:26:06 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:26:06.586575 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.571179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:39335 (local address 127.0.0.1:60252)
0811 22:26:06.571702 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.571752 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:06.572626 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:06.573479 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:06.573486 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:06.573832 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:06.574099 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:06.574112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.575878 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.575881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:06.576699 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:06.576710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.577595 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.577598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:06.577696 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:06.578247 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:06.578264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:06.581353 (+ 3089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:06.583343 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:06.583353 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:06.583366 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:06.583593 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:06.583905 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:06.583908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:06.583910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:06.584027 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:06.584649 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:06.584656 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:06.584807 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 22:26:06.584979 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 22:26:06.585457 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":162,"thread_start_us":151,"threads_started":1}
I20260811 22:26:06.586575 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.571181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.26.125.62:39335)
0811 22:26:06.571748 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:06.571765 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:06.571881 (+ 116us) server_negotiation.cc:407] Connection header received
0811 22:26:06.572972 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:06.572978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:06.573257 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:06.573474 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:06.574247 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.575724 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.576834 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.577426 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.577604 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:06.581509 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:06.581515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:06.581528 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:06.581553 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:06.583236 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:06.583699 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:06.583705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:06.583706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:06.583819 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:06.584110 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:06.584113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:06.584115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:06.584541 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:06.584676 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:06.585060 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 22:26:06.585399 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":449,"spinlock_wait_cycles":23296,"thread_start_us":160,"threads_started":1}
I20260811 22:26:06.589766 1751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:06.590649 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.590302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56916)
0811 22:26:06.590480 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.590562 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 22:26:06.590905 1660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
I20260811 22:26:06.591933 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.591715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56920)
0811 22:26:06.591824 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.591873 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:26:06.592090 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:43527: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab.
Entry for principal kudu/127.26.125.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab.
I20260811 22:26:06.597918 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.61:43527
--webserver_interface=127.26.125.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab
--principal=kudu/127.26.125.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:39335,127.26.125.61:43527,127.26.125.60:42605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:26:06.641373 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } attempt: 1
I20260811 22:26:06.642364 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.642044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56930)
0811 22:26:06.642209 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.642280 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:26:06.643268 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.643018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56942)
0811 22:26:06.643126 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.643182 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 22:26:06.643394 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:43527: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
I20260811 22:26:06.711530 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } attempt: 2
I20260811 22:26:06.712389 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.712098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56948)
0811 22:26:06.712242 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.712309 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 22:26:06.713232 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.712915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56962)
0811 22:26:06.713078 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.713150 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:26:06.713449 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.61:43527: Network error: Client connection negotiation failed: client connection to 127.26.125.61:43527: connect: Connection refused (error 111)
W20260811 22:26:06.720427 1759 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:26:06.720734 1759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:06.720767 1759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:06.720840 1759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:06.724977 1759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:06.725051 1759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:06.725073 1759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:06.725098 1759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:06.725132 1759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:06.730719 1759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:39335,127.26.125.61:43527,127.26.125.60:42605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.61.keytab
--principal=kudu/127.26.125.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.61:43527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.125.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:26:06.731943 1759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:06.733011 1759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:06.742108 1767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:06.742101 1765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:06.742203 1759 server_base.cc:1034] running on GCE node
W20260811 22:26:06.742133 1769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:06.743096 1759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:06.744236 1759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:06.745394 1759 hybrid_clock.cc:648] HybridClock initialized: now 1786487166745374 us; error 35 us; skew 500 ppm
Aug 11 22:26:06 dist-test-slave-5bn2 krb5kdc[1600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:06.748385 1759 init.cc:377] Logged in from keytab as kudu/127.26.125.61@KRBTEST.COM (short username kudu)
I20260811 22:26:06.749313 1759 webserver.cc:460] Webserver started at http://127.26.125.61:35385/ using document root <none> and password file <none>
I20260811 22:26:06.749886 1759 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:06.749948 1759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:06.750140 1759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:06.752240 1759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/data/instance:
uuid: "84c86569cf954c6aaf11c9ff393c3154"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "503014e315f2b143132d5a12bcec54b3"
server_key_iv: "9b5a330d25170d8e8be8eb68b2e9352a"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.752758 1759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/wal/instance:
uuid: "84c86569cf954c6aaf11c9ff393c3154"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "503014e315f2b143132d5a12bcec54b3"
server_key_iv: "9b5a330d25170d8e8be8eb68b2e9352a"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.756604 1759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:26:06.759173 1780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:06.760466 1759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:26:06.760596 1759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/wal
uuid: "84c86569cf954c6aaf11c9ff393c3154"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "503014e315f2b143132d5a12bcec54b3"
server_key_iv: "9b5a330d25170d8e8be8eb68b2e9352a"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.760706 1759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:06.777086 1759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:06.780510 1759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:06.780723 1759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:06.788335 1759 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.61:43527
I20260811 22:26:06.788646 1877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.61:43527 every 8 connection(s)
I20260811 22:26:06.789466 1759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/data/info.pb
I20260811 22:26:06.792534 1878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:06.796042 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1759
I20260811 22:26:06.796164 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-1/wal/instance
I20260811 22:26:06.796443 27124 external_mini_cluster.cc:1428] Setting key 7a1a3ec93fd89b693907703896c67e99
I20260811 22:26:06.796690 1878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:06.804617 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } attempt: 3
WARNING: no policy specified for kudu/127.26.125.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.125.60@KRBTEST.COM" created.
Aug 11 22:26:06 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
Aug 11 22:26:06 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
I20260811 22:26:06.815867 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.805087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56970)
0811 22:26:06.805403 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.805417 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:06.805484 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:06.806765 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:06.806769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:06.807003 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:06.807143 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:06.807151 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.808424 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.808436 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:06.809041 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:06.809051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.809778 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.809782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:06.809893 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:06.810573 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:06.810589 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:06.812040 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:06.814127 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:06.814130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:06.814132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:06.814371 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:06.814706 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:06.814709 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:06.814711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:06.814788 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:06.815396 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:06.815400 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:06.815500 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 22:26:06.815602 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:26:06.815735 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239}
I20260811 22:26:06.816010 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.798745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.26.125.62:39335)
0811 22:26:06.798943 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:06.798948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:06.799929 (+ 981us) server_negotiation.cc:407] Connection header received
0811 22:26:06.801097 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:06.801100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:06.801155 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:06.801229 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:06.802829 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.803396 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.804946 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.805264 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.805361 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:06.811222 (+ 5861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:06.811226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:06.811229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:06.811259 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:06.813378 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:06.814016 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:06.814020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:06.814022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:06.814112 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:06.814530 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:06.814533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:06.814535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:06.814773 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:06.814895 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:06.815677 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 22:26:06.815810 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60,"spinlock_wait_cycles":3840}
I20260811 22:26:06.817463 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.799189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:39335 (local address 127.0.0.1:60256)
0811 22:26:06.799742 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.799801 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:06.800856 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:06.801486 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:06.801497 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:06.802027 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:06.802671 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:06.802687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.803523 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.803526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:06.804781 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:06.804794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.805547 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.805552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:06.805757 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:06.806408 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:06.806443 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:06.811045 (+ 4602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:06.813485 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:06.813496 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:06.813510 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:06.813896 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:06.814200 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:06.814203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:06.814205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:06.814393 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:06.814904 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:06.814911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:06.815380 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 22:26:06.815917 (+ 537us) client_negotiation.cc:240] Negotiation successful
0811 22:26:06.816677 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":111,"thread_start_us":142,"threads_started":1}
I20260811 22:26:06.817461 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.805726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.26.125.61:43527)
0811 22:26:06.806170 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:06.806181 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:06.806208 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:26:06.806255 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:06.806261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:06.806487 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:06.807304 (+ 817us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:06.807434 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.808303 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.809160 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.809623 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.810173 (+ 550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:06.812209 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:06.812217 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:06.812235 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:06.812261 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:06.814035 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:06.814511 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:06.814518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:06.814520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:06.814613 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:06.814926 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:06.814930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:06.814932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:06.815293 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:06.815415 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:06.816270 (+ 855us) server_negotiation.cc:299] Negotiation successful
0811 22:26:06.816575 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":345,"thread_start_us":107,"threads_started":1}
I20260811 22:26:06.819168 1878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:06.820190 1751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:06.820899 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.820662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52128)
0811 22:26:06.820776 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.820828 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 22:26:06.821599 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.821358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52132)
0811 22:26:06.821490 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.821539 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 22:26:06.821762 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:42605: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Aug 11 22:26:06 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
I20260811 22:26:06.827000 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.819683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56984)
0811 22:26:06.819796 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.819812 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:06.819894 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:06.820105 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:06.820108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:06.820279 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:06.820423 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:06.820431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.821233 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.821236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:06.821797 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:06.821807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.822262 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.822266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:06.822376 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:06.822805 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:06.822816 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:06.824079 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:06.825502 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:06.825505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:06.825507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:06.825689 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:06.825988 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:06.825990 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:06.825992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:06.826066 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:06.826430 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:06.826434 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:06.826544 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 22:26:06.826638 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:26:06.826756 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 22:26:06.827039 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.819805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56984 (local address 127.26.125.61:43527)
0811 22:26:06.819902 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:06.819906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:06.819919 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:26:06.819956 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:06.819958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:06.820000 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:06.820074 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:06.820533 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.821076 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.821902 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:06.822162 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:06.822245 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:06.824202 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:06.824205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:06.824208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:06.824231 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:06.825403 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:06.825768 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:06.825771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:06.825773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:06.825853 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:06.826144 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:06.826148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:06.826149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:06.826325 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:06.826424 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:06.826683 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 22:26:06.826817 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":31}
I20260811 22:26:06.827854 1878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.125.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab.
Entry for principal kudu/127.26.125.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab.
I20260811 22:26:06.828714 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.828457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52144)
0811 22:26:06.828574 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.828656 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 22:26:06.829372 1788 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
I20260811 22:26:06.829536 27124 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
/tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.60:42605
--webserver_interface=127.26.125.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab
--principal=kudu/127.26.125.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.125.20:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.125.62:39335,127.26.125.61:43527,127.26.125.60:42605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:26:06.831552 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.831300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52156)
0811 22:26:06.831447 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.831497 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 22:26:06.831734 1878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:42605: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
I20260811 22:26:06.852975 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } attempt: 1
I20260811 22:26:06.853986 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.853621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52162)
0811 22:26:06.853821 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.853897 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:26:06.854781 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.854562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52176)
0811 22:26:06.854673 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.854720 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 22:26:06.854926 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:42605: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
I20260811 22:26:06.881044 1878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } attempt: 1
I20260811 22:26:06.882004 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.881662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52192)
0811 22:26:06.881846 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.881916 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 22:26:06.882743 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.882527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52198)
0811 22:26:06.882635 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.882682 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:26:06.882886 1878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:42605: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
I20260811 22:26:06.930117 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } attempt: 2
I20260811 22:26:06.931071 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.930694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52206)
0811 22:26:06.930896 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.930969 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:26:06.931828 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.931607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52222)
0811 22:26:06.931715 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.931764 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:26:06.931944 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:42605: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
W20260811 22:26:06.949549 1895 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:26:06.949913 1895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:06.949958 1895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:06.950016 1895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:26:06.951030 1878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } attempt: 2
I20260811 22:26:06.951857 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.951559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52232)
0811 22:26:06.951711 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.951775 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:26:06.952518 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:06.952310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52236)
0811 22:26:06.952412 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:06.952457 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 22:26:06.952626 1878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.125.60:42605: Network error: Client connection negotiation failed: client connection to 127.26.125.60:42605: connect: Connection refused (error 111)
W20260811 22:26:06.954392 1895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:06.954452 1895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:06.954480 1895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:06.954512 1895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:06.954536 1895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:06.960096 1895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.125.20:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.125.62:39335,127.26.125.61:43527,127.26.125.60:42605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/krb5kdc/kudu_127.26.125.60.keytab
--principal=kudu/127.26.125.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.125.60:42605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.125.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:26:06.961336 1895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:06.962487 1895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:06.971351 1903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:06.971446 1895 server_base.cc:1034] running on GCE node
W20260811 22:26:06.971470 1907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:06.971594 1902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:06.972234 1895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:06.973387 1895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:06.974567 1895 hybrid_clock.cc:648] HybridClock initialized: now 1786487166974544 us; error 32 us; skew 500 ppm
Aug 11 22:26:06 dist-test-slave-5bn2 krb5kdc[1600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:06.977557 1895 init.cc:377] Logged in from keytab as kudu/127.26.125.60@KRBTEST.COM (short username kudu)
I20260811 22:26:06.978484 1895 webserver.cc:460] Webserver started at http://127.26.125.60:43779/ using document root <none> and password file <none>
I20260811 22:26:06.979113 1895 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:06.979184 1895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:06.979364 1895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:06.981474 1895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/data/instance:
uuid: "2ae7e04104b04ecfa74fc340eaa62ddb"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "c7a4f246d103a157130ddd46cb9fea4b"
server_key_iv: "011bab40ad8dcaa0622d5959369c7ace"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.982034 1895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/wal/instance:
uuid: "2ae7e04104b04ecfa74fc340eaa62ddb"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "c7a4f246d103a157130ddd46cb9fea4b"
server_key_iv: "011bab40ad8dcaa0622d5959369c7ace"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.985970 1895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:26:06.988785 1918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:06.990190 1895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:06.990314 1895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/data,/tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/wal
uuid: "2ae7e04104b04ecfa74fc340eaa62ddb"
format_stamp: "Formatted at 2026-08-11 22:26:06 on dist-test-slave-5bn2"
server_key: "c7a4f246d103a157130ddd46cb9fea4b"
server_key_iv: "011bab40ad8dcaa0622d5959369c7ace"
server_key_version: "encryptionkey@0"
I20260811 22:26:06.990417 1895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:07.017236 1895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:07.020910 1895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:07.021087 1895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:07.028473 1895 rpc_server.cc:307] RPC server started. Bound to: 127.26.125.60:42605
I20260811 22:26:07.028752 2003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.125.60:42605 every 8 connection(s)
I20260811 22:26:07.029549 1895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/data/info.pb
I20260811 22:26:07.031119 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } attempt: 3
I20260811 22:26:07.033123 2007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:07.037358 2007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:07.037681 27124 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu as pid 1895
I20260811 22:26:07.037833 27124 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmkw4Fs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486868287078-27124-0/minicluster-data/master-2/wal/instance
I20260811 22:26:07.038232 27124 external_mini_cluster.cc:1428] Setting key ed8ed86cfb298b7d3927f76ce1b5c061
Aug 11 22:26:07 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.62@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
I20260811 22:26:07.043788 1878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } attempt: 3
I20260811 22:26:07.045185 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.031758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52248)
0811 22:26:07.031920 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:07.031938 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:07.032046 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:07.034693 (+ 2647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:07.034697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:07.034990 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:07.035126 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.035136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.036481 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.036485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.037027 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.037039 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.038038 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.038042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.038140 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.038830 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:07.038877 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:07.040579 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:07.042776 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.042779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.042782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.043116 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.043505 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.043508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.043510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.043607 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.044567 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:07.044571 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:07.044694 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 22:26:07.044805 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:26:07.044986 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
Aug 11 22:26:07 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
Aug 11 22:26:07 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:26:07.048554 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.032604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.26.125.60:42605)
0811 22:26:07.033132 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.033150 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.033191 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:26:07.033949 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.033968 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.034364 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.034732 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.035267 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.036325 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.037162 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.037875 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.038071 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.040752 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.040758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.040770 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.040802 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.042657 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.043231 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.043238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.043245 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.043410 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.043738 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.043740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.043742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.044440 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.044586 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.047157 (+ 2571us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.047497 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":157,"threads_started":1}
Aug 11 22:26:07 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.61@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
I20260811 22:26:07.053086 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.039033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.26.125.60:42605)
0811 22:26:07.040135 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.040144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.040160 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:26:07.040202 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.040205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.040299 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.040404 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.041462 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.042279 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.043453 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.043829 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.043918 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.048685 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.048689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.048692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.048721 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.050604 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.051185 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.051189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.051190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.051263 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.051572 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.051576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.051577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.051810 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.052031 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.052744 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.052872 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1024,"spinlock_wait_cycles":1868032,"thread_start_us":152,"threads_started":1}
I20260811 22:26:07.053145 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.039663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.26.125.62:39335)
0811 22:26:07.039783 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.039787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.040156 (+ 369us) server_negotiation.cc:407] Connection header received
0811 22:26:07.040396 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.040400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.040463 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.040509 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.041476 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.042279 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.043396 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.043751 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.043875 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.049229 (+ 5354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.049233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.049235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.049270 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.050991 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.051596 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.051601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.051603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.051705 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.052024 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.052028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.052030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.052331 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.052407 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.052873 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.052994 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:26:07.054092 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.039623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.62:39335 (local address 127.0.0.1:60270)
0811 22:26:07.040016 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:07.040078 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:07.040284 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:07.040667 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:07.040673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:07.041171 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:07.041306 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.041314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.042418 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.042422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.043279 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.043288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.044017 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.044021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.044160 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.045576 (+ 1416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:07.045594 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:07.049047 (+ 3453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:07.051124 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.051134 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.051141 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.051452 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.051817 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.051820 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.051821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.051915 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.052512 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:07.052518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:07.052647 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 22:26:07.053709 (+ 1062us) client_negotiation.cc:240] Negotiation successful
0811 22:26:07.053907 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":2,"spinlock_wait_cycles":5504,"thread_start_us":89,"threads_started":1}
I20260811 22:26:07.054523 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.044320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52266)
0811 22:26:07.044564 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:07.044579 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:07.044665 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:07.045146 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:07.045149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:07.045366 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:07.045487 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.045494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.046313 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.046317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.046938 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.046953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.047588 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.047592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.047690 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.048184 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:07.048200 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:07.051081 (+ 2881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:07.052997 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.053001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.053003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.053262 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.053579 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.053582 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.053584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.053669 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.054022 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:07.054025 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:07.054132 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 22:26:07.054217 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:26:07.054361 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 22:26:07.054548 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.044363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.26.125.60:42605)
0811 22:26:07.044941 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.044946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.044957 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:26:07.044985 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.044987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.045021 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.045095 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.045599 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.046206 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.047079 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.047446 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.047920 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.051235 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.051239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.051242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.051267 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.052872 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.053378 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.053381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.053382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.053477 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.053755 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.053759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.053760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.053920 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.054012 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.054278 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.054377 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":392,"threads_started":1}
I20260811 22:26:07.054876 1751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d: Bootstrap starting.
I20260811 22:26:07.055536 2007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:07.057893 1751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:07.058233 1878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154: Bootstrap starting.
I20260811 22:26:07.058995 1751 log.cc:824] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:26:07 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
I20260811 22:26:07.061231 1878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:07.061725 1751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d: No bootstrap required, opened a new log
I20260811 22:26:07.062227 1878 log.cc:824] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:07.065828 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.056052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.61:43527 (local address 127.0.0.1:56998)
0811 22:26:07.056282 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:07.056297 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:07.056392 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:07.056641 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:07.056643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:07.056816 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:07.056942 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.056949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.057898 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.057902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.058698 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.058712 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.059299 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.059304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.059412 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.059911 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:07.059926 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:07.061662 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:07.063884 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.063888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.063891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.064154 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.064509 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.064513 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.064515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.064639 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.065216 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:07.065220 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:07.065361 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 22:26:07.065473 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:26:07.065640 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260811 22:26:07.066543 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.056146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.26.125.61:43527)
0811 22:26:07.056265 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.056269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.056363 (+ 94us) server_negotiation.cc:407] Connection header received
0811 22:26:07.056484 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.056487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.056535 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.056612 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.057064 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.057770 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.058829 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.059178 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.059285 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.061884 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.061888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.061891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.061920 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.063752 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.064292 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.064296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.064297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.064391 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.064749 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.064753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.064755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.065054 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.065157 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.066249 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.066386 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:26:07.066509 1751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } }
I20260811 22:26:07.066793 1751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f324d607bc4043b2f82983af4a622d, State: Initialized, Role: FOLLOWER
Aug 11 22:26:07 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.62@KRBTEST.COM
I20260811 22:26:07.067203 1751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } }
I20260811 22:26:07.067354 2007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:26:07.067795 2019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.068053 2019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:07.068406 1878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154: No bootstrap required, opened a new log
I20260811 22:26:07.068868 1751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:07.071514 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.061042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.26.125.62:39335)
0811 22:26:07.061165 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.061169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.061494 (+ 325us) server_negotiation.cc:407] Connection header received
0811 22:26:07.061544 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.061547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.061595 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.061657 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.062546 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.063298 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.064251 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.064527 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.064648 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.067780 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.067784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.067788 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.067819 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.069373 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.069872 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.069876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.069877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.069988 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.070337 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.070341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.070343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.070559 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.070722 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.071167 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.071327 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:26:07.072760 1878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } }
I20260811 22:26:07.073015 1878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c86569cf954c6aaf11c9ff393c3154, State: Initialized, Role: FOLLOWER
I20260811 22:26:07.073375 1878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } }
Aug 11 22:26:07 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.125.60@KRBTEST.COM for kudu/127.26.125.60@KRBTEST.COM
I20260811 22:26:07.075525 1878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:07.077622 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.067851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.125.60:42605 (local address 127.0.0.1:52274)
0811 22:26:07.068063 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:07.068076 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:07.068148 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:07.069590 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:07.069594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:07.069762 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:07.069881 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.069888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.070954 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.070957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.071715 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:07.071726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.072264 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.072267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:07.072349 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.072874 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:07.072886 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:07.074324 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:07.076158 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.076161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.076163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.076383 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.076662 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:07.076664 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:07.076666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:07.076735 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:07.077148 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:07.077151 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:07.077230 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 22:26:07.077303 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:26:07.077439 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
Aug 11 22:26:07 dist-test-slave-5bn2 krb5kdc[1600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.125.61@KRBTEST.COM
W20260811 22:26:07.081759 2056 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:26:07.081852 2056 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 22:26:07.082334 2039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:26:07.082409 2039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:26:07.074569 1621 master_runner.cc:186] Error getting master registration for 127.26.125.61:43527: OK, instance_id { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" instance_seqno: 1786487166786074 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:26:07.083455 1621 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:26:07.077790 2040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.083706 2040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:07.084270 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.068267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.26.125.60:42605)
0811 22:26:07.068700 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.068705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.068719 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:07.068775 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.068779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.068828 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.069974 (+ 1146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.070052 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.070806 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.071839 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.072149 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.072550 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.074447 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.074450 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.074452 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.074487 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.076051 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.076495 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.076498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.076499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.076577 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.076823 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.076826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.076827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.077052 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.083881 (+ 6829us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.083967 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.084106 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:26:07.084790 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.074090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57008 (local address 127.26.125.61:43527)
0811 22:26:07.074192 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.074197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.074209 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:26:07.074243 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.074245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.074287 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.074358 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.075218 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.075914 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.077125 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.077436 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.077514 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.080769 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.080773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.080776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.080806 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.082463 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.083008 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.083012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.083013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.083108 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.083414 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.083418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.083420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.083654 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.083813 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.084530 (+ 717us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.084653 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 22:26:07.084031 1759 master_runner.cc:186] Error getting master registration for 127.26.125.60:42605: OK, instance_id { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" instance_seqno: 1786487167026345 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:26:07.087366 1759 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:26:07.087487 2007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb: Bootstrap starting.
I20260811 22:26:07.090401 2007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:07.090448 27124 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:26:07.091430 2007 log.cc:824] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:07.091840 1700 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "52f324d607bc4043b2f82983af4a622d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60276
I20260811 22:26:07.092162 1700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:26:07.092291 1700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:07.093518 2007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb: No bootstrap required, opened a new log
I20260811 22:26:07.093693 1700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } }
I20260811 22:26:07.094810 1700 leader_election.cc:288] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [CANDIDATE]: Term 1 election: Requested vote from peers 84c86569cf954c6aaf11c9ff393c3154 (127.26.125.61:43527), 2ae7e04104b04ecfa74fc340eaa62ddb (127.26.125.60:42605)
I20260811 22:26:07.095818 1834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52f324d607bc4043b2f82983af4a622d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "84c86569cf954c6aaf11c9ff393c3154"
I20260811 22:26:07.096108 1834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:07.097535 1834 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52f324d607bc4043b2f82983af4a622d in term 1.
I20260811 22:26:07.098073 1660 leader_election.cc:302] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52f324d607bc4043b2f82983af4a622d, 84c86569cf954c6aaf11c9ff393c3154; no voters:
I20260811 22:26:07.098400 2019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:07.098671 2019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [term 1 LEADER]: Becoming Leader. State: Replica: 52f324d607bc4043b2f82983af4a622d, State: Running, Role: LEADER
I20260811 22:26:07.099102 2019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } }
I20260811 22:26:07.099172 2007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } }
I20260811 22:26:07.099443 2007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae7e04104b04ecfa74fc340eaa62ddb, State: Initialized, Role: FOLLOWER
I20260811 22:26:07.099828 2007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } }
I20260811 22:26:07.101213 2019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52f324d607bc4043b2f82983af4a622d. Latest consensus state: current_term: 1 leader_uuid: "52f324d607bc4043b2f82983af4a622d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.101346 2019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:07.101382 2007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:07.101456 1963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52f324d607bc4043b2f82983af4a622d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb"
I20260811 22:26:07.101727 1963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:07.103386 1963 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52f324d607bc4043b2f82983af4a622d in term 1.
I20260811 22:26:07.104759 2080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.104882 2080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [sys.catalog]: This master's current role is: FOLLOWER
W20260811 22:26:07.109324 2111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:26:07.109426 2111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:26:07.113704 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.103261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60284 (local address 127.26.125.62:39335)
0811 22:26:07.103381 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.103385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.104232 (+ 847us) server_negotiation.cc:407] Connection header received
0811 22:26:07.104444 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.104448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.104504 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.104563 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.105739 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.106433 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.107597 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.107892 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.107982 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.109997 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.110001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.110004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.110033 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.111784 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.112296 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.112300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.112302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.112392 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.112686 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.112690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.112692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.112910 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.113046 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.113404 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.113535 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:26:07.116794 2084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:07.117990 2084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:07.118562 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.104186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.26.125.60:42605)
0811 22:26:07.104570 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.104574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.106125 (+ 1551us) server_negotiation.cc:407] Connection header received
0811 22:26:07.107531 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.107534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.107581 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.107626 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.111080 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.111736 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.112638 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.112971 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.113065 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.114807 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.114811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.114813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.114842 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.116583 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.117069 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.117074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.117076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.117168 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.117474 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.117478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.117480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.117721 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.117869 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.118263 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.118394 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306}
I20260811 22:26:07.120299 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:07.104157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57016 (local address 127.26.125.61:43527)
0811 22:26:07.104283 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.104288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.105902 (+ 1614us) server_negotiation.cc:407] Connection header received
0811 22:26:07.106310 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.106314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.106366 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.106416 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.108335 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.109045 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.112670 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:07.112995 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:07.113065 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:07.116527 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:07.116531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:07.116534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:07.116566 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:07.118272 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.118742 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.118746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.118748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.118869 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:07.119302 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:07.119306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:07.119308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:07.119544 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:07.119677 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:07.120034 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 22:26:07.120155 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:26:07.125098 1963 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [term 1 FOLLOWER]: Refusing update from remote peer 52f324d607bc4043b2f82983af4a622d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:07.125876 2079 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:07.131484 2080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52f324d607bc4043b2f82983af4a622d. Latest consensus state: current_term: 1 leader_uuid: "52f324d607bc4043b2f82983af4a622d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.131632 2080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:07.133746 1834 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [term 1 FOLLOWER]: Refusing update from remote peer 52f324d607bc4043b2f82983af4a622d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:07.134410 2079 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:07.138672 2040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52f324d607bc4043b2f82983af4a622d. Latest consensus state: current_term: 1 leader_uuid: "52f324d607bc4043b2f82983af4a622d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.138819 2040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:07.143112 2079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52f324d607bc4043b2f82983af4a622d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.143270 2079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:07.144124 2115 mvcc.cc:205] Tried to move back new op lower bound from 7317451436535689216 to 7317451436442243072. Current Snapshot: MvccSnapshot[applied={T|T < 7317451436535689216}]
I20260811 22:26:07.145051 2080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52f324d607bc4043b2f82983af4a622d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.145179 2080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:07.145404 2118 mvcc.cc:205] Tried to move back new op lower bound from 7317451436535689216 to 7317451436442243072. Current Snapshot: MvccSnapshot[applied={T|T < 7317451436535689216}]
I20260811 22:26:07.154399 2019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52f324d607bc4043b2f82983af4a622d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.154572 2019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:07.155843 2040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52f324d607bc4043b2f82983af4a622d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f324d607bc4043b2f82983af4a622d" member_type: VOTER last_known_addr { host: "127.26.125.62" port: 39335 } } peers { permanent_uuid: "84c86569cf954c6aaf11c9ff393c3154" member_type: VOTER last_known_addr { host: "127.26.125.61" port: 43527 } } peers { permanent_uuid: "2ae7e04104b04ecfa74fc340eaa62ddb" member_type: VOTER last_known_addr { host: "127.26.125.60" port: 42605 } } }
I20260811 22:26:07.155972 2040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:07.156032 2084 catalog_manager.cc:1347] Generated new cluster ID: 9c8d63d3336e42ada75e1aa6ddde9c98
I20260811 22:26:07.156097 2084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:07.180913 2084 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:26:07.182313 2084 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:07.198637 2084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52f324d607bc4043b2f82983af4a622d: Generated new TSK 0
I20260811 22:26:07.199723 2084 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:26:07.227943 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.225040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.26.125.61:43527)
0811 22:26:07.225176 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.225180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.226502 (+ 1322us) server_negotiation.cc:407] Connection header received
0811 22:26:07.226996 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.227000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.227059 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.227116 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.227807 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57030: BlockingRecv error: recv got EOF from 127.0.0.1:57030 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:26:07.229341 2015 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.225811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.26.125.60:42605)
0811 22:26:07.225925 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.225929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.226894 (+ 965us) server_negotiation.cc:407] Connection header received
0811 22:26:07.228493 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.228496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.228564 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.228617 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.229194 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52294: BlockingRecv error: recv got EOF from 127.0.0.1:52294 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 22:26:07.231916 1755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.230396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.26.125.62:39335)
0811 22:26:07.230532 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.230536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.230890 (+ 354us) server_negotiation.cc:407] Connection header received
0811 22:26:07.231081 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.231084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.231136 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.231186 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.231790 (+ 604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60286: BlockingRecv error: recv got EOF from 127.0.0.1:60286 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 22:26:07.236930 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.235049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57034 (local address 127.26.125.61:43527)
0811 22:26:07.235163 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.235167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.235793 (+ 626us) server_negotiation.cc:407] Connection header received
0811 22:26:07.236032 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.236036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.236128 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.236186 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.236810 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57034: BlockingRecv error: recv got EOF from 127.0.0.1:57034 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 22:26:07.242442 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.240769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57038 (local address 127.26.125.61:43527)
0811 22:26:07.240884 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.240888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.241442 (+ 554us) server_negotiation.cc:407] Connection header received
0811 22:26:07.241660 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.241664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.241722 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.241775 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.242325 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57038: BlockingRecv error: recv got EOF from 127.0.0.1:57038 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 22:26:07.248817 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.246928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57046 (local address 127.26.125.61:43527)
0811 22:26:07.247057 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.247062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.247697 (+ 635us) server_negotiation.cc:407] Connection header received
0811 22:26:07.247918 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.247922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.247979 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.248037 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.248684 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57046: BlockingRecv error: recv got EOF from 127.0.0.1:57046 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 22:26:07.257833 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.255778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57048 (local address 127.26.125.61:43527)
0811 22:26:07.255945 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.255951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.256634 (+ 683us) server_negotiation.cc:407] Connection header received
0811 22:26:07.256880 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.256884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.256957 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.257021 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.257699 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57048: BlockingRecv error: recv got EOF from 127.0.0.1:57048 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 22:26:07.270464 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.268586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.26.125.61:43527)
0811 22:26:07.268712 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.268716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.269332 (+ 616us) server_negotiation.cc:407] Connection header received
0811 22:26:07.269557 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.269560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.269616 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.269669 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.270335 (+ 666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57060: BlockingRecv error: recv got EOF from 127.0.0.1:57060 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 22:26:07.290295 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.288943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57068 (local address 127.26.125.61:43527)
0811 22:26:07.289076 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.289080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.289360 (+ 280us) server_negotiation.cc:407] Connection header received
0811 22:26:07.289510 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.289514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.289568 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.289638 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.290164 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57068: BlockingRecv error: recv got EOF from 127.0.0.1:57068 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 22:26:07.325686 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.324443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57074 (local address 127.26.125.61:43527)
0811 22:26:07.324590 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.324595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.324705 (+ 110us) server_negotiation.cc:407] Connection header received
0811 22:26:07.324857 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.324860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.324924 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.325034 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.325554 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57074: BlockingRecv error: recv got EOF from 127.0.0.1:57074 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 22:26:07.393188 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.392065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57084 (local address 127.26.125.61:43527)
0811 22:26:07.392224 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.392228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.392325 (+ 97us) server_negotiation.cc:407] Connection header received
0811 22:26:07.392452 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.392454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.392506 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.392596 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.393068 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57084: BlockingRecv error: recv got EOF from 127.0.0.1:57084 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 22:26:07.524605 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.523463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57098 (local address 127.26.125.61:43527)
0811 22:26:07.523628 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.523633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.523772 (+ 139us) server_negotiation.cc:407] Connection header received
0811 22:26:07.523895 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.523897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.523950 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.524034 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.524431 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57098: BlockingRecv error: recv got EOF from 127.0.0.1:57098 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 22:26:07.784188 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:07.782936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57104 (local address 127.26.125.61:43527)
0811 22:26:07.783105 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:07.783109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:07.783131 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:26:07.783330 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:07.783333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:07.783391 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:07.783518 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:07.784059 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57104: BlockingRecv error: recv got EOF from 127.0.0.1:57104 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:26:08.083468 2056 catalog_manager.cc:1259] Loaded cluster ID: 9c8d63d3336e42ada75e1aa6ddde9c98
I20260811 22:26:08.083585 2056 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154: loading cluster ID for follower catalog manager: success
I20260811 22:26:08.085528 2056 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154: acquiring CA information for follower catalog manager: success
I20260811 22:26:08.086995 2056 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84c86569cf954c6aaf11c9ff393c3154: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:26:08.111004 2111 catalog_manager.cc:1259] Loaded cluster ID: 9c8d63d3336e42ada75e1aa6ddde9c98
I20260811 22:26:08.111115 2111 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb: loading cluster ID for follower catalog manager: success
I20260811 22:26:08.112867 2111 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb: acquiring CA information for follower catalog manager: success
I20260811 22:26:08.114085 2111 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2ae7e04104b04ecfa74fc340eaa62ddb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:26:08.303146 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:08.298566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.26.125.61:43527)
0811 22:26:08.298829 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:08.298833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:08.298884 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:26:08.299032 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:08.299035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:08.299172 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:08.299275 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:08.299684 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.300244 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.301264 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.301528 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.301636 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:08.302027 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:08.302560 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:08.302686 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:08.302894 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 22:26:08.302968 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260811 22:26:08.310947 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:08.306612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.26.125.60:42605)
0811 22:26:08.306897 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:08.306902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:08.306923 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:26:08.307110 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:08.307114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:08.307278 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:08.307405 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:08.307623 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.308481 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.309394 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.309640 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.309731 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:08.310002 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:08.310485 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:08.310633 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:08.310688 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 22:26:08.310749 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":83,"threads_started":1}
I20260811 22:26:08.318396 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:08.314358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.26.125.61:43527)
0811 22:26:08.314518 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:08.314523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:08.314937 (+ 414us) server_negotiation.cc:407] Connection header received
0811 22:26:08.315079 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:08.315082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:08.315135 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:08.315214 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:08.315461 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.316074 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.317168 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.317507 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.317605 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:08.317877 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:08.317970 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:08.318040 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:08.318186 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 22:26:08.318253 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:26:08.318763 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:08.314616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.26.125.60:42605)
0811 22:26:08.314774 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:08.314778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:08.315064 (+ 286us) server_negotiation.cc:407] Connection header received
0811 22:26:08.315114 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:08.315117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:08.315174 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:08.315234 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:08.315623 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.316401 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.317464 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.317771 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.317862 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:08.318169 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:08.318251 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:08.318309 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:08.318421 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 22:26:08.318473 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:26:08.319476 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:08.314612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.26.125.62:39335)
0811 22:26:08.314868 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:08.314873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:08.314887 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:08.314996 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:08.314999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:08.315178 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:08.315295 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:08.315738 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.316527 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.317489 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.317776 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.317873 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:08.318300 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:08.318776 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:08.318902 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:08.319270 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 22:26:08.319336 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":68,"threads_started":1}
I20260811 22:26:08.323019 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1621
I20260811 22:26:08.341305 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1759
I20260811 22:26:08.351140 27124 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmkw4Fs/build/debug/bin/kudu with pid 1895
2026-08-11T22:26:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3308 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6773 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (300068 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST